Skip to content

Paged attention unit tests - #102

Open
Aditya Jadhav (aditjadh) wants to merge 5 commits into
qualcomm:mainfrom
aditjadh:paged-attention-unit-tests
Open

Aditya Jadhav (aditjadh) wants to merge 5 commits into
qualcomm:mainfrom
aditjadh:paged-attention-unit-tests

Conversation

@aditjadh

Copy link
Copy Markdown
Contributor

Adds an automated test suite for Paged Attention that validates:

Test Coverage

  1. Correctness across varying prompt lengths and KV block boundaries
  2. Output consistency across different runtime configurations
  3. Batch invariance guarantees
  4. Prefix caching functionality in multi-chat scenarios
  5. Prefix caching performance improvements for long-context workloads
  6. Model accuracy with and without prefix caching enabled
  7. Multimodal (vision-language) support with Paged Attention

Models Covered
Text Models : meta-llama/Llama-3.1-8B-Instruct
Multimodal Models : Qwen/Qwen2.5-VL-3B-Instruct

Signed-off-by: Aditya Dhananjay Jadhav <aditjadh@qti.qualcomm.com>
Signed-off-by: Aditya Dhananjay Jadhav <aditjadh@qti.qualcomm.com>
Signed-off-by: Aditya Dhananjay Jadhav <aditjadh@qti.qualcomm.com>
Signed-off-by: Aditya Dhananjay Jadhav <aditjadh@qti.qualcomm.com>
Signed-off-by: Aditya Dhananjay Jadhav <your-email@qualcomm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant