Support conditional prefill based on actual decode kv cache state

[elevran]

Enable conditional disaggregated prefill based on actual cache state. vLLM can return a missing cache response which indicates to the P/D sidecar that a prefill should be used.

See proposal in this doc and related vLLM issue

cc: @nilig @kfirwolfson

[kyanokashi]

Can I take a go at this?

[elevran]

hi @kyanokashi - this is not ready in vLLLM yet (see issue linked in description above).
Until that's resolved, we would have no way to test the feature here or ensures it conforms with the settings that would be eventually adapted.
So I think it is premature to start coding it.

[kfirwolfson]

PR 24520 for vLLM is ready for some time and is working well. It's possible to take a fork and test with it, so no blocker there. I believe once we have this implemented, both PRs will get prioritized and reviewed.

[elevran]

Based on above comment by @kfirwolfson - /assign @kyanokashi