Skip to content

Rebase vLLM to v0.28.0 - #98

Open
Sanidhya Singal (quic-sanising) wants to merge 13 commits into
qualcomm:dev/v0.28.0from
quic-sanising:v0.28.0
Open

Sanidhya Singal (quic-sanising) wants to merge 13 commits into
qualcomm:dev/v0.28.0from
quic-sanising:v0.28.0

Conversation

@quic-sanising

@quic-sanising Sanidhya Singal (quic-sanising) commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Rebases the QAIC plugin from vLLM v0.23.0 to v0.28.0.

Version bumps

  • setup.py, pyproject.toml, scripts/utility.sh: v0.23.0 → v0.28.0
  • requirements/vllm_dependency_{aot,pyt}.txt: aligned with upstream v0.28.0 (transformers, fastapi/starlette, prometheus-fastapi-instrumentator, xgrammar, mistral_common, jsonschema; dropped diskcache/gguf)
  • requirements/lint.txt: pre-commit>=4.5.1

v0.28.0 API adaptations

  • patch_parallel_state.py: add use_all2all arg to QaicGroupCoordinator (new GroupCoordinator param)
  • worker/model_runner.py: unpack new _num_nans_device from _bookkeeping_sync; set self.pin_memory = False (upstream moved to module-level PIN_MEMORY); patch torch.cuda.Event in _torch_cuda_wrapper
  • patch_block_table.py: handle new SlotMappingMode (early-return for NONE)
  • patch_rejection_sampler.py: pass synthetic_mode/synthetic_conditional_rates/use_fp64_gumbel to rejection_sample; drop stale skip-softmax path
  • patch_mem_utils.py: capability-guard platform memory hooks + add MemorySnapshot.__post_init__ override
  • __init__.py: apply graph-pickler Options shim at import (torch < 2.8 compat)

Transformers 5.15.1 compatibility

  • qaic_custom_mm_processor.py: fallback import for Qwen2VLImageProcessorFast; widen version gate to
    >= 5.5.4

Test / CI

  • test_qaic_lora.py: fix swapped seq_len/ctx_len (violated new long_prefill_token_threshold ≤ max_model_len check)
  • requirements/test.txt: consolidate test deps (drop linters, add datasets)
  • ci_pipeline_run_plugin.sh: install test deps from requirements/test.txt (single source of truth)

Validation

  • examples/qaic.py runs end-to-end in AOT
  • AOT CI (ci_fasttest_qaic_aot.sh) passes

Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
@ankitqti

Copy link
Copy Markdown
Contributor

Create a new branch "dev/v0.28.0" and make PR on that

Did you ran ci_scripts/ci_fasttest_qaic_aot.sh?

@quic-sanising

Copy link
Copy Markdown
Contributor Author

Create a new branch "dev/v0.28.0" and make PR on that

Did you ran ci_scripts/ci_fasttest_qaic_aot.sh?

Ankit Arora (@ankitqti) The PR is not ready yet. I'll let you know once it is ready.

@quic-sanising
Sanidhya Singal (quic-sanising) changed the base branch from main to dev/v0.28.0 August 28, 2026 17:36
@quic-sanising

Copy link
Copy Markdown
Contributor Author

Create a new branch "dev/v0.28.0" and make PR on that

Did you ran ci_scripts/ci_fasttest_qaic_aot.sh?

[scheduler] 15/15 job(s) passed
[scheduler] total execution time: 417.8s (0:06:58)

Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
@ankitqti

Copy link
Copy Markdown
Contributor

Update vLLM version in ,md files if needed in documentation folder

Signed-off-by: Sanidhya Singal <sanising@qti.qualcomm.com>
@quic-sanising

Copy link
Copy Markdown
Contributor Author

Update vLLM version in ,md files if needed in documentation folder

Done. How do we manage release details and repo structure in docs/docs/community/release_notes.md and docs/STRUCTURE.md?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants