Skip to content

fix(vllm): reset block hash on prefix-cache eviction#363

Open
shipiyouniao wants to merge 1 commit into
ovg-project:mainfrom
shipiyouniao:fix/prefix-block-hash-reset
Open

fix(vllm): reset block hash on prefix-cache eviction#363
shipiyouniao wants to merge 1 commit into
ovg-project:mainfrom
shipiyouniao:fix/prefix-block-hash-reset

Conversation

@shipiyouniao

Copy link
Copy Markdown

Summary

Fixes stale vLLM KVCacheBlock.block_hash state when kvcached evicts or frees prefix-cache blocks.

Closes #362.

Root Cause

ElasticBlockPool removes evicted blocks from kvcached's _cached_blocks, _block_id_to_key, and _evictable_blocks indexes, but the underlying vLLM block object can keep its old block_hash value. Reusing that block for a different prefix can leave stale block state behind, and newer vLLM block implementations require reset_hash() before assigning a new hash.

What Changed

  • Add _reset_block_hash() helper that calls reset_hash() when available and falls back to clearing _block_hash for older/mock block objects.
  • Reset block hash when blocks are evicted from the lazy prefix-cache pool.
  • Reset block hash for uncached blocks that are immediately returned to kvcached.
  • Reset block hash during explicit evict_blocks() and reset_prefix_cache().
  • Add regression coverage to tests/test_prefix_cache.py for eviction, reset, explicit eviction, and reallocation/recache flows.

Tests

python -m pytest tests/test_prefix_cache.py -q

Result: 28 passed.

Copilot AI review requested due to automatic review settings June 24, 2026 14:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a correctness issue in the vLLM integration where KVCacheBlock.block_hash could remain stale after prefix-cache eviction/free, causing reused blocks to carry incorrect hash state across prefixes.

Changes:

  • Add a _reset_block_hash() helper to clear vLLM block hash state (preferring reset_hash() when available).
  • Invoke block-hash reset on eviction/free/reset flows in the patched ElasticBlockPool.
  • Extend tests/test_prefix_cache.py to regress block-hash clearing across eviction, explicit eviction, reset, and reuse scenarios.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
kvcached/integration/vllm/patches.py Clears KVCacheBlock hash state when blocks are returned to kvcached via eviction/free/reset paths to prevent stale reuse.
tests/test_prefix_cache.py Strengthens the mock block to enforce write-once hash semantics and adds assertions ensuring hashes are cleared on eviction/free/reset/reuse.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread kvcached/integration/vllm/patches.py
@shipiyouniao shipiyouniao force-pushed the fix/prefix-block-hash-reset branch 2 times, most recently from 7c90873 to ab1cdab Compare June 25, 2026 08:19
Comment thread kvcached/integration/vllm/patches.py
@shipiyouniao shipiyouniao force-pushed the fix/prefix-block-hash-reset branch from ab1cdab to c1b6999 Compare June 25, 2026 08:38
@shipiyouniao shipiyouniao force-pushed the fix/prefix-block-hash-reset branch from c1b6999 to 3ade28d Compare July 6, 2026 16:28
@shipiyouniao shipiyouniao force-pushed the fix/prefix-block-hash-reset branch from 3ade28d to 25b735f Compare July 6, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

vLLM prefix cache blocks keep stale block_hash after eviction

3 participants