Skip to content

Commit d58bab0

Browse files
EngHabuEngHabu
authored
Fix cargo build and add CI check (#6645)
* Fix cargo build and add CI check Signed-off-by: Haytham Abuelfutuh <[email protected]> * lint lib.rs Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update Readme and contributing guides Signed-off-by: Haytham Abuelfutuh <[email protected]> * signed? Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix build target Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix build-crate target Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix included ts files in npm Signed-off-by: Haytham Abuelfutuh <[email protected]> --------- Signed-off-by: Haytham Abuelfutuh <[email protected]>
1 parent 84d3182 commit d58bab0

File tree

10 files changed

+581
-65
lines changed

10 files changed

+581
-65
lines changed

‎.github/workflows/check-generate.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,4 +44,7 @@ jobs:
4444
- name: Check for changes
4545
run: |
4646
git diff --exit-code || (echo 'Generated files are out of date. Run make generate and commit changes.' && exit 1)
47+
- name: Check rust builds
48+
run: |
49+
make build-crate
4750

‎CONTRIBUTING.md‎

Lines changed: 366 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,366 @@
1-
# Contribution Guidelines
2-
3-
## Contributing to Idl
4-
To contribute to the IDL, please follow these steps:
5-
1. Fork the repository.
6-
2. Make your changes in a new branch.
7-
3. Run `make buf` to generate idl changes.
8-
4. Commit (and Sign) your changes with a clear message.
9-
5. Push your changes to your fork.
10-
6. Create a pull request against the `main` branch of the original repository.
11-
12-
### Releasing
13-
1. Ensure that your changes are merged into the `main` branch.
14-
2. Create a new release by strictly following the SemVer 2.0.0 guidelines.
15-
This guarantees Python, Rust, NPM, and Go versions are in sync.
1+
# Contributing to Flyte 2
2+
3+
Thank you for your interest in contributing to Flyte 2! This guide will help you understand the contribution workflow, testing requirements, and release process.
4+
5+
## Table of Contents
6+
- [Getting Started](#getting-started)
7+
- [Development Workflow](#development-workflow)
8+
- [Making Changes](#making-changes)
9+
- [Testing and Verification](#testing-and-verification)
10+
- [Submitting Changes](#submitting-changes)
11+
- [Release Process](#release-process)
12+
- [Best Practices](#best-practices)
13+
14+
## Getting Started
15+
16+
### Prerequisites
17+
18+
Before contributing, ensure you have:
19+
- [Buf CLI](https://buf.build/docs/installation) installed
20+
- Go 1.24.6 or later
21+
- Node.js and npm (for TypeScript)
22+
- Python 3.9+ with `uv` package manager
23+
- Rust toolchain (if working with Rust bindings)
24+
- Git configured with your name and email
25+
26+
### Setting Up Your Environment
27+
28+
1. **Fork the repository** on GitHub
29+
30+
2. **Clone your fork**:
31+
```bash
32+
git clone https://github.com/YOUR_USERNAME/flyte.git
33+
cd flyte/flyte
34+
```
35+
36+
3. **Add upstream remote**:
37+
```bash
38+
git remote add upstream https://github.com/flyteorg/flyte.git
39+
```
40+
41+
4. **Install required tools**:
42+
```bash
43+
make download_tooling
44+
```
45+
46+
5. **Verify your setup**:
47+
```bash
48+
make gen
49+
```
50+
51+
## Development Workflow
52+
53+
### Creating a Feature Branch
54+
55+
Always create a new branch for your changes:
56+
57+
```bash
58+
git checkout -b feature/your-feature-name
59+
```
60+
61+
Use descriptive branch names:
62+
- `feature/add-new-task-type` - for new features
63+
- `fix/workflow-literal-bug` - for bug fixes
64+
- `docs/improve-readme` - for documentation
65+
- `refactor/simplify-interface` - for refactoring
66+
67+
### Keeping Your Branch Updated
68+
69+
Regularly sync with upstream:
70+
71+
```bash
72+
git fetch upstream
73+
git rebase upstream/v2
74+
```
75+
76+
## Making Changes
77+
78+
### 1. Modifying Protocol Buffers
79+
80+
When editing `.proto` files in `flyteidl2/`:
81+
82+
**Naming Conventions:**
83+
- Use `snake_case` for field names: `task_id`, `execution_time`
84+
- Use `PascalCase` for message names: `TaskDefinition`, `WorkflowSpec`
85+
- Use `SCREAMING_SNAKE_CASE` for enum values: `TASK_STATE_RUNNING`
86+
87+
**Backward Compatibility:**
88+
- Never change field numbers
89+
- Never change field types
90+
- Never remove required fields
91+
- Use `reserved` for removed fields:
92+
```protobuf
93+
message Example {
94+
reserved 2, 15, 9 to 11;
95+
reserved "old_field_name";
96+
}
97+
```
98+
99+
**Documentation:**
100+
- Add clear comments for all messages, fields, and enums
101+
- Include examples where helpful
102+
- Document any constraints or validation rules
103+
104+
**Example:**
105+
```protobuf
106+
// TaskDefinition defines the structure and configuration of a task.
107+
message TaskDefinition {
108+
// Unique identifier for the task
109+
string task_id = 1;
110+
111+
// Human-readable name for the task
112+
string name = 2;
113+
114+
// Optional description explaining the task's purpose
115+
string description = 3;
116+
}
117+
```
118+
119+
### 2. Generate and Format Code
120+
121+
After making changes:
122+
123+
```bash
124+
# Format proto files
125+
make buf-format
126+
127+
# Lint proto files
128+
make buf-lint
129+
130+
# Generate all language bindings
131+
make buf
132+
```
133+
134+
### 3. Update Language-Specific Utilities (if needed)
135+
136+
If you need to customize generation for a specific language, update files in `flyteidl2/gen_utils/`:
137+
- `flyteidl2/gen_utils/go/` - Go-specific utilities
138+
- `flyteidl2/gen_utils/ts/` - TypeScript utilities
139+
- `flyteidl2/gen_utils/python/` - Python package configuration
140+
- `flyteidl2/gen_utils/rust/` - Rust crate configuration
141+
142+
## Testing and Verification
143+
144+
### 1. Lint and Format Checks
145+
146+
```bash
147+
make buf-lint
148+
make buf-format
149+
```
150+
151+
All proto files must pass linting without errors.
152+
153+
### 2. Verify Generated Code Builds
154+
155+
**Go:**
156+
```bash
157+
make buf-go
158+
make go-tidy
159+
cd gen/go
160+
go build ./...
161+
go test ./...
162+
```
163+
164+
**TypeScript:**
165+
```bash
166+
make buf-ts
167+
cd gen/ts
168+
npm install
169+
npm run build # if there's a build script
170+
```
171+
172+
**Python:**
173+
```bash
174+
make buf-python
175+
cd gen/python
176+
uv lock
177+
uv sync
178+
```
179+
180+
**Rust:**
181+
```bash
182+
make buf-rust
183+
make build-crate
184+
```
185+
186+
### 3. Generate Mocks (Go)
187+
188+
If you've modified Go interfaces:
189+
190+
```bash
191+
make mocks
192+
```
193+
194+
### 4. Integration Testing
195+
196+
Test your changes in a downstream project that uses flyte:
197+
1. Use `replace` directive in `go.mod` to point to your local changes
198+
2. Generate code and verify it works as expected
199+
3. Run integration tests in the consuming project
200+
201+
## Submitting Changes
202+
203+
### 1. Commit Your Changes
204+
205+
Write clear, descriptive commit messages:
206+
207+
```bash
208+
git add .
209+
git commit -s -m "feat: add new task execution state
210+
211+
- Add TASK_STATE_CACHED for cached task results
212+
- Update state transitions to support caching
213+
- Add documentation for caching behavior
214+
215+
Fixes #123"
216+
```
217+
218+
**Commit message format:**
219+
- Use conventional commits: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`
220+
- Include `-s` flag to sign your commits (required)
221+
- Reference issue numbers with `Fixes #123` or `Closes #456`
222+
223+
### 2. Push Your Changes
224+
225+
```bash
226+
git push origin feature/your-feature-name
227+
```
228+
229+
### 3. Create a Pull Request
230+
231+
1. Go to the [Flyte repository](https://github.com/flyteorg/flyte)
232+
2. Click "New Pull Request"
233+
3. Select your fork and branch
234+
4. Target the `v2` branch (not `main`)
235+
5. Fill out the PR template with:
236+
- Description of changes
237+
- Motivation and context
238+
- Testing performed
239+
- Screenshots (if UI changes)
240+
- Breaking changes (if any)
241+
242+
### 4. Code Review Process
243+
244+
- Address reviewer feedback promptly
245+
- Push new commits to the same branch
246+
- Use `git commit --amend` for small fixes (then `git push --force`)
247+
- Engage in constructive discussion
248+
- Be patient - reviews may take time
249+
250+
## Release Process
251+
252+
Releases are created by maintainers following these steps:
253+
254+
### Version Management
255+
256+
Flyte 2 follows [Semantic Versioning 2.0.0](https://semver.org/):
257+
258+
- **MAJOR** version (v2.0.0 → v3.0.0): Breaking changes
259+
- **MINOR** version (v2.1.0 → v2.2.0): New features, backward compatible
260+
- **PATCH** version (v2.1.1 → v2.1.2): Bug fixes, backward compatible
261+
262+
### Creating a Release (Maintainers Only)
263+
264+
1. **Ensure all changes are merged into `v2` branch**
265+
266+
2. **Determine the version number** based on changes:
267+
```bash
268+
# View changes since last release
269+
git log v2.X.Y..HEAD --oneline
270+
```
271+
272+
3. **Create and push a tag**:
273+
```bash
274+
git checkout v2
275+
git pull upstream v2
276+
git tag -a v2.X.Y -m "Release v2.X.Y"
277+
git push upstream v2.X.Y
278+
```
279+
280+
4. **Create GitHub Release**:
281+
- Go to [Releases](https://github.com/flyteorg/flyte/releases)
282+
- Click "Draft a new release"
283+
- Select the tag you just created
284+
- Generate release notes
285+
- Add any additional context or highlights
286+
- Publish release
287+
288+
5. **Verify Artifacts**:
289+
The release triggers automated publishing to:
290+
- Go modules: `github.com/flyteorg/flyte/v2`
291+
- NPM: `@flyteorg/flyte`
292+
- PyPI: `flyteidl2`
293+
- Crates.io: `flyte`
294+
295+
### Post-Release
296+
297+
1. **Verify published packages**:
298+
```bash
299+
# Go
300+
GOPROXY=https://proxy.golang.org go list -m github.com/flyteorg/flyte/[email protected]
301+
302+
# NPM
303+
npm view @flyteorg/[email protected]
304+
305+
# PyPI
306+
pip index versions flyteidl2
307+
308+
# Crates.io
309+
cargo search flyte
310+
```
311+
312+
2. **Update dependent projects** with the new version
313+
314+
3. **Announce the release** in community channels
315+
316+
## Best Practices
317+
318+
### Protocol Buffer Design
319+
320+
- **Keep messages small and focused** - Single responsibility principle
321+
- **Use oneof for mutually exclusive fields**:
322+
```protobuf
323+
message Task {
324+
oneof task_type {
325+
PythonTask python = 1;
326+
ContainerTask container = 2;
327+
}
328+
}
329+
```
330+
- **Use optional for truly optional fields** (proto3)
331+
- **Use repeated for arrays/lists**
332+
- **Provide sensible defaults** where appropriate
333+
334+
### Documentation
335+
336+
- Document all public APIs
337+
- Include usage examples in comments
338+
- Keep README.md and CONTRIBUTING.md up to date
339+
- Add inline comments for complex logic
340+
341+
### Git Workflow
342+
343+
- Commit early and often
344+
- Keep commits atomic and focused
345+
- Write meaningful commit messages
346+
- Squash small "fix" commits before PR
347+
- Rebase instead of merge when updating your branch
348+
349+
### Communication
350+
351+
- Be respectful and professional
352+
- Ask questions when unclear
353+
- Provide context in discussions
354+
- Update PR descriptions as scope changes
355+
- Respond to reviews in a timely manner
356+
357+
## Getting Help
358+
359+
- **Documentation**: [Flyte Documentation](https://docs.flyte.org)
360+
- **Community Slack**: [Flyte Slack](https://slack.flyte.org)
361+
- **GitHub Issues**: [Report Issues](https://github.com/flyteorg/flyte/issues)
362+
- **GitHub Discussions**: [Ask Questions](https://github.com/flyteorg/flyte/discussions)
363+
364+
## License
365+
366+
By contributing to Flyte 2, you agree that your contributions will be licensed under the Apache License 2.0.

‎Makefile‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,3 +99,7 @@ gen: buf mocks go_tidy ## Generates everything in the 'gen' directory
9999
@echo '⚡ Finished generating everything in the gen directory'
100100
@$(MAKE) sep
101101

102+
build-crate: ## Builds the rust crate
103+
@echo 'Cargo build the generated rust code'
104+
cd gen/rust && cargo build
105+
@$(MAKE) sep

0 commit comments

Comments
 (0)