Problem
Index Load/Deserialize (and Release) cannot be canceled once started. For a large vector index (e.g. multi-GB DiskANN or HNSW), deserialize/load runs for tens of seconds to minutes with no way to abort.
This hurts Milvus after milvus-io/milvus#52478: QueryCoord now enforces a task timeout on segment load and propagates cancellation via gRPC → QueryNode → segcore. Milvus segcore already threads a folly::CancellationToken through the load path (OpContext), and the field-data / scalar-index paths check it at fine granularity. But the vector-index path dead-ends at knowhere:
VectorMemIndex::Load → knowhere Deserialize(BinarySet) — no cancellation parameter, runs to completion.VectorDiskIndex::Load → knowhere DeserializeFromFile / DiskANN load — same.
So when the coordinator gives up (default 5 min), the QueryNode goroutine and the segcore thread stay pinned inside knowhere until the load finishes on its own; the retried load then queues behind resources the dead load still holds (memory, load-pool slot, disk IO bandwidth).
Proposal
Accept a cancellation signal in the load/release entry points, e.g.:
- add an optional
folly::CancellationToken (knowhere already depends on folly) to Index::Deserialize / DeserializeFromFile / Load, or expose it via Config/LoadConfig;
- check the token at natural chunk boundaries: per binary-set entry, per file/slice read, per DiskANN sector-cache warm-up batch, per graph-partition during HNSW/IVF deserialize;
- on cancellation, return a distinct error (e.g.
Status::cancelled) and leave no partially-registered state (caller frees the partially built index);
- same for
Release/destructor-heavy paths if they can block for long (DiskANN file cleanup), though load is the priority.
No behavior change when the token is not passed.
Motivation summary
Coordinated timeout/cancel is only as good as its weakest link; today knowhere index load is the longest non-cancellable section in the Milvus segment-load path (minutes-scale, bounded only by index size / S3 + disk bandwidth).
/kind enhancement
Problem
Index
Load/Deserialize(andRelease) cannot be canceled once started. For a large vector index (e.g. multi-GB DiskANN or HNSW), deserialize/load runs for tens of seconds to minutes with no way to abort.This hurts Milvus after milvus-io/milvus#52478: QueryCoord now enforces a task timeout on segment load and propagates cancellation via gRPC → QueryNode → segcore. Milvus segcore already threads a
folly::CancellationTokenthrough the load path (OpContext), and the field-data / scalar-index paths check it at fine granularity. But the vector-index path dead-ends at knowhere:VectorMemIndex::Load→ knowhereDeserialize(BinarySet)— no cancellation parameter, runs to completion.VectorDiskIndex::Load→ knowhereDeserializeFromFile/ DiskANN load — same.So when the coordinator gives up (default 5 min), the QueryNode goroutine and the segcore thread stay pinned inside knowhere until the load finishes on its own; the retried load then queues behind resources the dead load still holds (memory, load-pool slot, disk IO bandwidth).
Proposal
Accept a cancellation signal in the load/release entry points, e.g.:
folly::CancellationToken(knowhere already depends on folly) toIndex::Deserialize/DeserializeFromFile/Load, or expose it viaConfig/LoadConfig;Status::cancelled) and leave no partially-registered state (caller frees the partially built index);Release/destructor-heavy paths if they can block for long (DiskANN file cleanup), though load is the priority.No behavior change when the token is not passed.
Motivation summary
Coordinated timeout/cancel is only as good as its weakest link; today knowhere index load is the longest non-cancellable section in the Milvus segment-load path (minutes-scale, bounded only by index size / S3 + disk bandwidth).
/kind enhancement