[ci] Refactor CI files to use new CI pipeline generator #246
Conversation
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - label: "Diffusion Model Test" | ||
| timeout_in_minutes: 15 | ||
| commands: | ||
| - pytest -s -v tests/single_stage/test_diffusion_model.py |
There was a problem hiding this comment.
The GPU test steps are now plain command invocations without the docker or Kubernetes plugins that previously ran them inside public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT with HF cache mounts. With this change they execute directly on the host even though the build job still builds and pushes the container, so any GPU agent lacking the full Python environment or pre-seeded HuggingFace cache (common in these pipelines) will fail as soon as pytest starts because the required dependencies/models are missing.
Useful? React with 👍 / 👎.
Signed-off-by: Kevin H. Luu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
khluu
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| - pytest -s -v tests/multi_stages/ | ||
|
|
||
| - label: "Omni Model Test with H100" | ||
| timeout_in_minutes: 20 |
There was a problem hiding this comment.
We used to set the timeout to 15 minutes. @ywang96 Do you agree we set 20 minutes for testing on H100?
There was a problem hiding this comment.
The timeout is already 20 minutes on main branch https://github.com/vllm-project/vllm-omni/blob/main/.buildkite/pipeline.yml#L55
There was a problem hiding this comment.
The timeout is already 20 minutes on main branch https://github.com/vllm-project/vllm-omni/blob/main/.buildkite/pipeline.yml#L55
Oops, my mistake! Thanks for catching that.
| gpu: h100 | ||
| num_gpus: 2 | ||
| commands: | ||
| - export VLLM_WORKER_MULTIPROC_METHOD=spawn |
There was a problem hiding this comment.
Do we also need to set the logging level here, align with the Omni Model Test?
There was a problem hiding this comment.
Is this empty file mandatory for the Buildkite test?
There was a problem hiding this comment.
Ya it's used as an indicator whether a branch has the new refactored changes or not, to route CI bootstrap step to use the correct pipeline generator. The new pipeline generator wouldn't work with the old yaml file, and vice versa.
There was a problem hiding this comment.
Ya it's used as an indicator whether a branch has the new refactored changes or not, to route CI bootstrap step to use the correct pipeline generator. The new pipeline generator wouldn't work with the old yaml file, and vice versa.
Thanks for the elaboration, very clear.
| no_plugin: true | ||
|
|
||
| - label: "Diffusion Model Test" | ||
| timeout_in_minutes: 15 |
There was a problem hiding this comment.
Is the timeout applied per label or per command?
|
I plan to merge this once we migrate |
Change structure & format of CI files to use new vLLM project Buildkite pipeline generator https://github.com/vllm-project/ci-infra/tree/main/buildkite/pipeline_generator