docs: add Idempotent Requests user guide

[bigsheeper]

Draft of a user guide for the idempotency key introduced by milvus-io/milvus#52544 (bulk import) and milvus-io/milvus#50007 (insert).

This PR adds one markdown file and nothing else. Placement and linking are left to a doc engineer. The page is not registered in menuStructure/en.json and nothing links to it, so merged as-is it is reachable only by direct link. Which section it belongs to, what it is called in the nav, and which pages should point at it are decisions I did not want to make unilaterally. Earlier revisions of this PR made them; those commits are reverted.

What the page covers

• Sending the key: REST header Idempotency-Key, gRPC metadata idempotency-key, with pymilvus, Go SDK and curl examples.
• Choosing a key: one per logical request, retry with the same key, what reuse actually does, length and character bounds, no secrets.
• Insert: both switches and the error if you send a key too early, what a retry looks like, keys Milvus derives when you send none, and how far back a shard remembers.
• Bulk import: the lost-jobId problem, collection scoping across renames, and the two cases that need a new key.
• A pointer to the configuration reference rather than a parameter table.

Scope

This is a user guide, deliberately not a specification. It went through four adversarial review rounds that pulled it toward spec completeness, and at 2283 words a reader who only wanted a safe retry was reading about per-channel byte budgets, tombstone re-stamping across restarts, a length floor on a parameter they will never set, and DDL duplicate-rejection hazards. All true, none of it useful here. The page was cut back to 986 words in 5406ad0e.

The standard I would ask reviewers to apply: judge it by whether a reader can act on it, not by whether every sentence is complete. Precision that changes no reader decision belongs in the parameter docs and the design doc, which already carry it. The bar for adding something back is a concrete user action that goes wrong without it.

Status of the underlying features

• Bulk import honors the key on servers with enhance: Support idempotent broadcast, with BulkImport as its first user milvus#52544.
• Insert honors it once enhance: Support idempotent write milvus#50007 lands.
• pymilvus support is in enhance: support idempotent requests for DML pymilvus#3784, still open. The page says SDK support is pending release rather than citing versions that do not exist.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fjo8onCWrFYYkzsECo1DHN

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bigsheeper

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[czs007]

Must-fix issues introduced by this PR: 1
Merge recommendation: Mergeable after the 1 must-fix item below is fixed in this PR.

This documentation review of the new idempotent-requests page found one high-severity gap and several accuracy issues, all introduced by this PR, with the most important being that the page does not warn readers that keyless inserts are content-hash deduplicated once idempotent insert is enabled.

High

• site/en/userGuide/insert-and-delete/idempotent-requests.md:120 — The page never warns that, once streaming.idempotency.enabled=true and the collection property are on, a keyless insert gets an automatic content-derived key and is deduplicated by payload. With both switches on and an autoID collection, two intentionally distinct inserts whose db/collection/partition/numRows/field payloads are byte-identical within the same shard window → the second insert returns the first insert's result (same insert_count and autoIDs) and writes nothing, so the second record is silently lost with no error. Line 24 ("A request without a key behaves exactly as before") is false in this configuration, and the server-side property comment says such workloads MUST NOT enable the property or must supply distinct explicit keys; this also conflicts with the PR's own edit to insert-update-delete.md ("creates a new entity ... data duplication"). Suggestion: scope the "unchanged behavior" claim to collections where idempotency is disabled, and state that distinct logical inserts with identical payloads need distinct explicit keys; only retries should reuse a key. (raised by czs007, sijie-ni-0214)

Medium

• site/en/userGuide/insert-and-delete/idempotent-requests.md:181 — Turning off the global switch and restarting wipes the persisted insert dedup window on all shards, and the page does not say so. An operator sets streaming.idempotency.enabled=false and restarts (or a channel fails over) while the store holds records, then re-enables, and a client whose insert response was lost before the toggle retries with the original key → every persisted summary chunk for the pchannel was deleted at WAL open, so the retry is treated as a fresh write and the rows are inserted a second time with new autoIDs and no error. This contradicts line 126's "an outage does not empty the window" and is broader than DropPartition, yet is absent from "Cases to know" at line 130. Suggestion: add to the config table row that turning it off and restarting drops every retained insert record on all shards and re-enabling starts from an empty window, and add "The global switch was turned off" to Cases to know. (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:59 — "An operation not in this table accepts a well-formed key and ignores it" is only true for pymilvus and REST; the Go SDK rejects an idempotency key on Upsert. A Go SDK user calls client.Upsert with WithIdempotencyKey("k") on either the column-based or row-based option → the request returns ErrParameterInvalid ("idempotency key is only supported for Insert") client-side, no RPC is sent, and the upsert never executes. The design doc documents this as intended. Suggestion: document the per-SDK difference instead of a blanket "accepts and ignores". (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:132 — "The original insert failed. Nothing landed and the key was released." is presented as unconditional, but the design doc's Known Limitations state that some WAL implementations (Pulsar explicitly) may persist the write despite returning an error. On a Pulsar WAL, the original keyed insert's append returns an error after the message was actually persisted and the client retries with the same key → the released key is re-owned by the retry, which appends again, so the rows land twice with different autoIDs and no error. Suggestion: qualify the statement to note this ambiguous-append case. (raised by czs007)

Low

• site/en/userGuide/insert-and-delete/idempotent-requests.md:180 — "0 rejects every key" for streaming.idempotency.maxKeyLength is only true where ValidateIdempotencyKey runs (REST middleware and the coordinator-client unary interceptor); the proxy insert path checks limit > 0 && len(key) > limit, so 0 means unbounded there. With maxKeyLength=0, a gRPC insert with an explicit key on a collection whose metadata is already cached at the proxy → the insert is accepted and deduplicated by key, while the identical request after a cache miss returns error 1100 from the coordinator-client interceptor. The default of 256 in the table is correct and matches the current server default. Suggestion: reword the row to say 0 rejects keys on REST and on any request that reaches a coordinator, while the proxy insert path treats 0 as unbounded. (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:66 — "Keep it under 256 bytes" contradicts the maxKeyLength=256 row in the configuration table; both length checks are len(key) > limit, so a key of exactly 256 bytes is accepted. Suggestion: change to "up to 256 bytes" or "at most 256 bytes". (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:120 — "field order" is listed as something that makes payloads distinct for the automatic key, but canonicalInsertPayloadKey sorts fields by name before hashing, so two keyless inserts differing only in field order produce the same automatic key. Suggestion: remove "field order" from the list; row order and encoding remain valid. (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:174 — "The default retention is sized so that this does not happen at default settings" overstates the tombstone protection; the default covers at most one StreamingCoord restart per tombstone lifetime, and milvus.yaml says to raise it further for more. With default taskRetention=172800 and tombstone.maxLifetime=24h, StreamingCoord restarts twice within one tombstone lifetime and an orchestrator retries the import with the original key after the finished job has been GC'd but while the twice-extended tombstone is still alive → the retry returns the original jobId and the subsequent describe call reports that the job does not exist, so the orchestrator cannot track the import. Suggestion: state the condition (at most one StreamingCoord restart per tombstone lifetime) and cross-reference the taskRetention row, which line 187 already describes correctly. (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:1 — The page states no minimum pymilvus, Go SDK, or server version for idempotency-key support. pymilvus support is in pymilvus#3784 (still open) and Go SDK support ships with milvus#50007 (still open, param table tags the feature as version 2.6.6), while this docs PR targets v3.0.x. Readers on older SDKs will not get the described behavior: pymilvus silently ignores an unknown kwarg, and older Go SDK has no WithIdempotencyKey. Suggestion: add a version requirements note. (raised by czs007)

[bigsheeper]

Thanks, all nine were real. Fixed in 3d5e61cd.

High. The page contradicted itself: the transport section claimed a keyless request behaves as before, while the Insert section said Milvus derives a key from the payload. Scoped the first claim to collections with idempotency off, and added a note that two intentionally distinct inserts with identical payloads collapse into one, with the two ways out: an explicit key per insert, or leave the collection property off.

Medium. Added a "The global switch was turned off" case and a matching clause in the config table. Replaced the blanket "accepts and ignores" with the Go SDK's client-side rejection on Upsert. Qualified the failed-original case with the ambiguous-append behavior.

Low. Explained that maxKeyLength=0 rejects at the REST door and the coordinator but reads as unbounded on the proxy insert path. Changed "under 256 bytes" to "at most 256". Dropped "field order", since fields are sorted before hashing. Replaced the retention claim with its actual condition, one StreamingCoord restart per tombstone lifetime. Added a note that an older SDK sends no key.

Two things I left alone deliberately: the pre-existing "Duplicate handling" bullet in insert-update-delete.md, which is outside this PR's scope, and the "Safe retries" bullet this PR adds there, which is accurate as written.

[czs007]

Must-fix issues introduced by this PR: 0
Merge recommendation: Mergeable as-is — this review found no must-fix issue introduced by this PR.

Adversarial review found no issues requiring changes.

This round re-checked the fixes landed in commit 3d5e61cd9c for the issues raised earlier on this PR, and all of them hold.

Verified:

• site/en/userGuide/insert-and-delete/idempotent-requests.md:66 — the "behaves exactly as before" claim for keyless requests is now conditioned on the collection switch: unchanged behavior only when idempotency is off, content-based deduplication when idempotent insert is enabled, with a pointer to the Explicit and automatic keys section.
• site/en/userGuide/insert-and-delete/idempotent-requests.md:164 — the new alert note states that two distinct inserts with the same payload are treated as one (second call returns the first call's result, writes nothing, raises no error), which matches the server-side property contract and closes the earlier high-severity finding.
• site/en/userGuide/insert-and-delete/idempotent-requests.md:168 — the note gives both viable workarounds (a distinct explicit key per insert, or leaving the collection property off), so readers with legitimately duplicate payloads have an actionable path.
• site/en/userGuide/insert-and-delete/idempotent-requests.md (range fd8a312a800f..HEAD) — the remaining wording corrections from the earlier round (global-switch-off clearing the dedup window, Go SDK Upsert rejecting a key, the failed-original-request exception, maxKeyLength=0 semantics, the 256-byte bound, field normalization, the import window, older SDKs not sending keys) were each checked against the referenced server behavior and introduce no new critical or high issues.

No new findings survived verification, and no previously raised item is outstanding.

[czs007]

Must-fix issues introduced by this PR: 2
Merge recommendation: Mergeable after the 2 must-fix items below are fixed in this PR.

Re-review of 3d5e61cd9c66

No findings were posted on this exact commit in an earlier pass, so there is nothing to confirm or retract here. Of the nine items raised on this PR in the previous round (on an earlier commit), reviewers verified that eight are resolved in this commit. The remaining one, the missing SDK version numbers, is carried below.

Overview: this round finds two internal contradictions in the new guide's description of its own idempotency model, plus the still-open SDK-version gap.

High

• site/en/userGuide/insert-and-delete/idempotent-requests.md:13 — The opening note says an older client's keyless request "behaves as a normal, non-idempotent one" with no qualifier, but line 30 says a keyless insert on a collection with idempotent insert enabled is deduplicated by its content, and line 130 states the second identical keyless insert writes nothing and raises no error. Failure: a legacy producer intentionally inserts the same byte-identical batch twice on a collection with both streaming.idempotency.enabled and collection.insert.idempotency.enabled set → the second insert returns the first result and writes nothing, so only one batch lands while the doc's first paragraph told the operator this could not happen. Suggestion: scope the opening note to distinguish collections with insert idempotency enabled vs. disabled, the same way line 30 is scoped. (raised by sijie-ni-0214, tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:148 — Lines 70 ("A reused key returns the old result and silently skips the new work") and 148 ("Either way, the new rows are not written") describe key reuse as a complete no-op, but the page's own model at line 136 says the key deduplicates per shard: shards that already hold the key return the original result and shards that never received it apply the write. Failure: a producer reuses key K (originally used for batch A on a 2-shard collection, all rows hashed to shard 1) for a different batch B with the same row count and PK type but rows hashing to shard 2 → shard 2 applies B's rows and the call reports success with A's ids, so an operator following lines 70/148 either re-sends B under a fresh key and creates duplicates that are unfindable by primary key under autoID, or never re-sends and believes B is absent when part of it is present. This is derived from the doc's stated per-shard model, not verified against server code. Suggestion: either confirm the server rejects or skips on every shard and fix line 136 accordingly, or scope lines 70 and 148 to the shard and state that key reuse can leave a partial write that must be reconciled, e.g. "shards that already hold the key return the original result and write nothing, while shards the original insert never reached apply the new rows." (raised by tinswzy)

Low

• site/en/userGuide/insert-and-delete/idempotent-requests.md:13 — The line names the API surface (idempotency_key, WithIdempotencyKey) but gives no minimum pymilvus or Go SDK version, so release availability cannot be established from the page. This was raised in the previous round; the author replied that all nine items were fixed in 3d5e61cd, but the version numbers are still absent at head. The likely reason is that pymilvus support is not yet released (enhance: support idempotent requests for DML pymilvus#3784 is still open), so there may be no number to cite yet. Suggestion: add the minimum pymilvus and Go SDK versions once the SDK releases land, or note explicitly that support is pending release. (raised by sijie-ni-0214, tinswzy)

[bigsheeper]

Agreed on all three. Fixed in 7806a3e0.

High, the client-support note. This was my own regression: fixing the previous round's version-note item reintroduced the very unscoped claim that round had removed from the transport section. Now scoped the same way, so an older SDK's keyless insert on an idempotency-enabled collection is described as still getting an automatic content-derived key.

High, key reuse. Confirmed against the server code, not only the doc's model, so this can be treated as verified rather than derived:

• The key is stamped per shard message, and buildSingleInsertMessageForStreamingService emits nothing for a shard with no rows, so a shard the original insert never reached never sees the key.
• warnOnPartialIdempotentDuplicate exists to log exactly the duplicates != 0 && duplicates != total case, which is only reachable when some shards dedup and others write.
• The design doc in enhance: Support idempotent write milvus#50007 states it under Known Limitations: a retry that reached only some shards is "deduplicated on the landed shards and appended fresh on the missing ones".
• The reuse error does not catch it. mergeInsertIDsByOffsets fails only on an id-type mismatch or an offset beyond the new result, so a batch with the same row count and primary key type merges silently and the call returns success.

The page now describes that outcome and narrows when the error is actually raised. I kept the per-shard behavior as the model rather than flattening it, since it is what lets a retry after a partial failure complete the write.

Low, SDK versions. You are right that my previous reply overclaimed: I added a note naming the required API surface, not the versions. Since milvus-io/pymilvus#3784 is still open there is no number to cite, so the note now says support is pending release and that this page will name the minimum versions once they ship.

[czs007]

Must-fix issues introduced by this PR: 0
Merge recommendation: Mergeable as-is — this review found no must-fix issue introduced by this PR.

Adversarial review of this docs-only PR found one medium and five low documentation issues in the new idempotency guide and its cross-link; no blocking correctness problems in the server-side claims that could be checked.

Medium

• site/en/userGuide/insert-and-delete/insert-update-delete.md:17 — The new "Safe retries" bullet tells the reader to send an idempotency key but never mentions that both streaming.idempotency.enabled and the collection property default to off. A reader on a default cluster follows the bullet and calls client.insert(..., idempotency_key="...") → the proxy rejects every call with ErrParameterInvalid (code 1100, non-retriable), turning a working insert into a deterministic hard failure. The parallel bullet in import-data.md does not have this problem because bulk import does not depend on the switch. Suggestion: add "…after enabling idempotent insert on the collection (off by default)". (raised by xiaocai2333, tinswzy)

Low

• site/en/userGuide/insert-and-delete/idempotent-requests.md:114 — The insert walkthrough (steps 1–4) only covers the "write completed, response lost" branch, yet step 2 names a client-side timeout as the trigger, which does not imply the server finished. The import section states the in-flight case explicitly at :178, but the insert section is silent. The server behavior is defined: the per-shard window tracks in-flight keys, a same-key retry that arrives while the original append is pending waits for it and then returns the original result, and if the original append fails the waiter receives that error and the key is released. Suggestion: add one sentence mirroring :178, e.g. "A retry that arrives while the original insert is still being written waits for it and then returns its result; if the original fails, the retry returns that error." (raised by tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:72 — The blanket rule says Milvus "does not compare the retry's body against the original and does not reject a reused key", while the Insert section at :150 says Milvus raises "idempotency key was reused with a different payload" when the original result cannot be mapped onto the new request (different primary key type, fewer rows). That error is an ErrParameterInvalid (1100) response, i.e. a rejection, and it is deterministic on every retry. Suggestion: scope the general rule, e.g. "…and rejects a reused key only in the narrow shape-mismatch cases described under Insert". (raised by xiaocai2333)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:140 — "There is no time limit, so an outage does not empty the window" holds only if no other writer touches the shard. The window is one object per vchannel, shared by all writers, and eviction is write-triggered (oldest-first by commit order on every commit). Writer A loses the response for key K and is down while other writers push more than streaming.idempotency.maxBytesPerWindow (16 MiB default) of insert records through the same shard, then A retries with K → K has been evicted, the retry is applied as a fresh write, and the rows exist twice with new autoIDs and no error. This is distinct from the per-pchannel maxRetainedBytes bound: it needs only a second writer and no restart. Suggestion: state that the window is shared by all writers to the shard, so an outage preserves it only while the shard is otherwise idle or below the byte cap. (raised by tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:201 — streaming.walBroadcaster.tombstone.maxLifetime and .maxCount are declared with Export: false in pkg/util/paramtable/component_param.go on milvus master and do not appear as keys in configs/milvus.yaml, unlike dataCoord.import.taskRetention, which is exported. They are still honored if written by hand, but an operator following the "keep it at least twice maxLifetime" guidance who searches milvus.yaml for the key finds nothing. Suggestion: mark both rows as absent from the default yaml (must be added explicitly), or move maxCount to prose. (raised by tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:178 — "so the returned jobId always refers to a job that exists" conflicts with :190, which says a retry after task cleanup returns the original jobId whose describe call reports that the job does not exist. Suggestion: scope the "always" to the registration race, e.g. "so the returned jobId refers to a job that was registered" or "so a retry never observes a half-registered job". (raised by xiaocai2333)

[bigsheeper]

All six addressed in 4556ed6d. I agree with five as stated; on the reuse-rejection item I took a different fix, with reasoning below.

Medium, the Insert Entities cross-link. Agreed, and this is the one I would have wanted caught before merge even though the round is non-blocking: a reader on a default cluster following that bullet gets a deterministic 1100. The bullet now says idempotent insert must be enabled on the collection, that it is off by default, and that sending a key beforehand is rejected. I had told an earlier round I was leaving this bullet alone as "accurate as written". It was accurate and incomplete in a way that misleads, which is worse in a doc than an error the reader can see.

Where I diverge: the reuse-rejection contradiction. The suggestion was to scope the general rule to "rejects a reused key only in the narrow shape-mismatch cases". The ordering in task_insert_streaming.go argues against that word:

resp := streaming.WAL().AppendMessagesWithOptions(...)   // writes happen here
if err := resp.UnwrapFirstError(); err != nil { ...; return nil }
it.result.Timestamp = resp.MaxTimeTick()
if it.idempotencyEnabled {
    warnOnPartialIdempotentDuplicate(...)
    if err := mergeDuplicateInsertResults(it.result, resp); err != nil {
        it.result.Status = merr.Status(merr.WrapErrParameterInvalidMsg("idempotency key was reused with a different payload; ..."))
    }
}

The 1100 is set after the append already succeeded, so nothing is refused. Calling it a rejection would tell a reader that a 1100 means nothing was written, when the rows on the non-duplicate shards are already in the WAL. That is the more dangerous misreading and it compounds the partial-write case fixed in the previous round. The general rule now says Milvus "does not refuse the request", and the insert case adds that the error is reported after the writes have gone through, so it is a warning that the two requests disagree rather than a sign that nothing landed.

The other four. Added the in-flight sentence to the insert walkthrough, mirroring the import section. Scoped the window claim: the window is one object per shard shared by every writer and eviction is commit triggered, so other writers' traffic can evict a key during an outage. Marked both tombstone rows as absent from the shipped yaml, which I confirmed in the milvus tree (Export: false on both, and configs/milvus.yaml has a walBroadcaster: section with no tombstone: subsection, while dataCoord.import.taskRetention is Export: true). Replaced "always refers to a job that exists" with "never observes a half-registered job", which is what the sentence was actually about.

[czs007]

Must-fix issues introduced by this PR: 1
Merge recommendation: Mergeable after the 1 must-fix item below is fixed in this PR.

Adversarial review of the new idempotent-requests guide and its cross-links found seven accuracy issues in the added text, all concentrated in the bulk-import window semantics, the insert window's durability scope, and the configuration table; no earlier-round item is re-raised, since all twelve were resolved in 3d5e61cd, 7806a3e0, and 4556ed6d.

High

• site/en/userGuide/insert-and-delete/idempotent-requests.md:186 — "up to 24 hours" states the tombstone window as an upper bound, but it is a lower bound. Recovered tombstones get createTime: time.Now() on every StreamingCoord start (tombstone_scheduler.go:51-56) and GC measures age from that (:113-127), so each restart extends every live key by up to another maxLifetime; the taskRetention parameter doc says the same (component_param.go:7221-7226). Line 186 therefore contradicts the page's own correct statements at :192 and :205. Failure: an operator sets dataCoord.import.taskRetention equal to tombstone.maxLifetime (24h) on the strength of line 186, StreamingCoord restarts while a key is live, and the orchestrator retries after the job is GC'd but before the extended tombstone expires → CreateImport returns the original jobId (services.go:1978-1982 does not check job existence), describe reports the job missing, the orchestrator records success, and the rows are never imported. Suggestion: "at least 24 hours by default; a StreamingCoord restart extends every live key by up to another maxLifetime". (raised by tinswzy, czs007)

Medium

• site/en/userGuide/insert-and-delete/idempotent-requests.md:142 — The restart-survival promise is scoped to the shard, but the durable budget is per physical channel. The paragraph scopes every sharing and eviction caveat to the shard (vchannel) while promising the durable copy "survives a restart or a failover", yet the table at :201-202 and component_param.go:8962/8972 scope maxRetainedBytes / maxRetainedChunks per pchannel, and window.go:119-122,151-152 confirm the store budget is per pchannel while the window is per vchannel. A pchannel carries vchannels of many collections (pool of rootCoord.dmlChannelNum=16 by default). Failure: idle collection A and heavy-insert collection B share a pchannel, B's writes push retained bytes past 256 MiB or chunks past maxRetainedChunks, a streaming node restart or failover follows, and a client of A retries an insert whose response was lost → A's key is no longer in the recovered window, the retry is applied as a fresh insert, and under autoID the duplicate rows carry new primary keys and cannot be found by the IDs returned to the original call. Suggestion: state that the durable copy is bounded per physical channel and shared with every collection on that channel, so the restart window can be shorter than the in-memory one. (raised by tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:198 — The maxKeyLength row omits the 64-byte floor imposed by the automatic key. At #50007 head 67e1e51, task_insert_idempotency.go:89 checks limit > 0 && len(it.idempotencyKey) > limit after the auto key is assigned at :84, and canonicalInsertPayloadKey returns hex.EncodeToString(sha256) (:437), i.e. 64 bytes. The row's current warning only covers retries whose explicit key exceeds a lowered limit; the floor bites the opposite population. Failure: streaming.idempotency.maxKeyLength set to any value in 1..63 on a cluster with the global switch on and a collection with idempotent insert enabled, and any client inserts without an explicit key → every keyless insert to that collection is rejected at the proxy with error 1100 idempotency key length 64 exceeds limit N, and old clients that cannot send a key have no workaround. Suggestion: add that the value must stay >= 64 while any collection has idempotent insert enabled. (raised by czs007, tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:190 — "This is the one import case where a new key is correct" is wrong in two independent directions. (1) GC'd original: datacoord/services.go:1978-1982 returns the original jobId without checking whether the job still exists, so a retry with the same key after cleanup never makes progress until the tombstone expires; the page's own line 192 describes this case, so a new key is also required there. (2) Live original: ddl_callbacks_import.go:96-105 documents that validateImportRequest (including ValidateMaxImportJobExceed at :122) runs before the idempotency lookup, so a retry can be rejected while the original job is alive, and there the correct action is to retry the same key later. Failure: retry with the original key after the job has been removed by taskRetention but while the tombstone still holds the key → every retry returns a jobId that describe reports missing, so an orchestrator following "only a failed job justifies a new key" loops until the tombstone expires and the data is never imported; separately, retry while the import job limit is saturated → the retry gets a validation error and an orchestrator following the sentence mints a new key, creating a second job that imports the files twice. Suggestion: replace the absolute with "new key after a failed job and after a GC'd job; same key, retried later, after a validation rejection". (raised by tinswzy, czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:203 — The tombstone parameter rows do not say they are the DDL duplicate-rejection store. Rows :203-204 describe streaming.walBroadcaster.tombstone.maxLifetime / maxCount purely as the bulk-import window, while the parameter docs at component_param.go:8744-8746 and :8755-8757 state the tombstone store exists to reject duplicate DDL submissions and that too few tombstones may lead to ABA issues in cluster state. Failure: an operator who never sends import keys lowers maxLifetime / maxCount well below defaults to "shrink the import window", following the table's description → the same store stops rejecting re-delivered DDL broadcasts sooner, exposing the cluster to the ABA hazard the page gives no hint of. Suggestion: add one clause to each row, or a note under the table, saying these parameters also bound DDL duplicate rejection. (raised by tinswzy, czs007)

• site/en/userGuide/insert-and-delete/insert-update-delete.md:17 — The cross-link bullet attributes enablement to the collection property alone, which is insufficient. The guide requires both the cluster switch streaming.idempotency.enabled and the collection property (idempotent-requests.md:83) and rejects a keyed insert with 1100 if either is off (:103); component_param.go:8942 states the same. The earlier fix in 4556ed6d added "which is off by default" and the rejection sentence, but left the attribution to the collection untouched. Failure: on a default cluster the user follows the bullet, sets only collection.insert.idempotency.enabled=true, and sends inserts with idempotency_key → every such insert is rejected with error 1100, and the user has done exactly what the bullet describes as enabling the feature. Suggestion: "Enabling idempotent insert, which requires a cluster setting plus a collection property and is off by default, lets you send…". (raised by tinswzy)

Low

• site/en/userGuide/insert-and-delete/idempotent-requests.md:198 — The "not present in the default milvus.yaml" note applies to all five streaming.idempotency rows, not just the two tombstone rows. maxKeyLength, enabled, maxBytesPerWindow, maxRetainedBytes, and maxRetainedChunks are all Export: false at #50007 head 67e1e51 (component_param.go:8772, 8945, 8955, 8965, 8975), and #50007 changes no file under configs/, so none of them appear in the default milvus.yaml. Suggestion: move the "add it explicitly" note from rows :203-204 to a single sentence covering the whole table. (raised by tinswzy, czs007)

[bigsheeper]

Thanks. Rather than fix these one by one, I rewrote the page in 5406ad0e. It is now 986 words, down from 2283. Five of the seven findings concern text that no longer exists.

Why the rewrite

Four review rounds each found statements that were incomplete, and each fix added a qualifier. That is the correct standard for a design doc, and applying it to a user guide produced a page where someone who only wants a safe insert retry had to read about per-channel byte budgets, tombstone re-stamping across restarts, a length floor on a parameter they will never set, and DDL duplicate-rejection hazards. Every one of those statements is true. I verified them all. None of them belong on this page.

The root problem was that the page was three documents at once: a user guide, a configuration reference, and a semantics spec. The spec parts invited spec-level scrutiny, so each round found more holes and the page grew another layer of hedging.

The principle I would ask later reviewers to apply

Judge this page by whether a reader can act on it, not by whether every sentence is complete.

Concretely:

• A statement that is incomplete but leads the reader to the right action is better here than a complete one they will not finish reading. Precision that changes no decision belongs in the parameter docs or the design doc, and both already carry it.
• Operator-facing detail is out of scope: internal topology (vchannel versus pchannel), GC and retention mechanics, parameter interaction rules, and failure modes that only a cluster admin can cause or fix.
• The bar for adding something back is a concrete user action that goes wrong without it. "A reader might infer X" is not enough; "a reader who follows this bullet breaks their inserts" is.

If a future round finds a removed statement missing, that is expected. A shorter sentence is by construction less complete. Please weigh it against the bar above before asking for it back.

What I did keep from this round

• The cross-link attribution. Fixed, and it was the one finding in this round that traps a real user: the bullet credited enablement to the collection property alone, so a reader could enable half of it and get error 1100 on every insert. It now names both the cluster setting and the property.
• The import "one case" absolute. Gone. The page now lists both cases that need a new key, the failed job and the collected job. I deliberately did not re-add the job-limit rejection case; it was removed earlier as an unlikely scenario, and the guidance to retry the same key already covers it.
• The window direction. The page no longer states a bound in either direction. It says the import key is remembered for about a day, nominal rather than guaranteed, shortened by traffic and lengthened by a restart. That is what a client can act on, without the tombstone mechanics.

What I dropped on purpose

The per-channel versus per-shard durable budget, the 64-byte floor on maxKeyLength, the tombstone store's DDL duplicate-rejection role, the 0 versus unbounded split on maxKeyLength, and the eight-row configuration table. The table is replaced by a pointer to the configuration reference. These are correct and they are operator material.

[czs007]

Must-fix issues introduced by this PR: 0
Merge recommendation: Mergeable as-is — this review found no must-fix issue introduced by this PR.

This review of the new idempotent-requests.md page found three medium and three low documentation-accuracy issues, all in text introduced by this PR; the enablement, key-format, keyless-dedup, and import-retention sections were checked against the server code and hold.

Medium

• site/en/userGuide/insert-and-delete/idempotent-requests.md:59 — The "Turn it on" section presents streaming.idempotency.enabled: true as a setting you add and then use, but the streaming node reads this flag only once, when the WAL interceptor is built (builder.go:21), and passes every append through while that captured value is false (idempotency_interceptor.go:93-95). The proxy reads the flag on every request (task_insert_idempotency.go:63-67), so it stops returning 1100 as soon as the file is reloaded. Operator flips the flag on a running cluster, sets the collection property, and starts sending keys without restarting streaming nodes → the proxy accepts the key while the WAL performs no dedup, and a same-key retry inserts the rows a second time, silently. Suggestion: add a note that the cluster setting is read when the streaming node opens its WAL, and that streaming nodes (or standalone) must be restarted after changing it before relying on the key. (raised by czs007)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:140 — The only configuration guidance is "See the system configuration reference for the full list and defaults", while the same paragraph says none of the streaming.idempotency.* keys appear in the default milvus.yaml. All five parameters are Export: false in component_param.go, and configure_streaming.md in this repo contains no idempotency entries, so the pointer resolves to nothing. A reader follows the pointer to find the window size or key-length default → the reference has no streaming.idempotency entries, so the parameter names and defaults cannot be discovered from the docs at all. Suggestion: replace the pointer with an inline key/default table: enabled=false, maxKeyLength=256, maxBytesPerWindow=16777216, maxRetainedBytes=268435456, maxRetainedChunks=256. (raised by czs007, xiaocai2333)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:89 — "If the original failed, nothing landed and the retry writes normally" is an unconditional guarantee the server does not make. The interceptor's own KNOWN LIMITATION comment (idempotency_interceptor.go:191-203) says some WAL implementations (pulsar walimpls) can persist the write while returning an error, and the key is released on that error. Original keyed insert on a pulsar-backed WAL returns an append error after the message was actually persisted, and the client retries with the same key → the retry re-owns the key and appends again under a fresh timetick, producing duplicate rows. Suggestion: qualify the sentence, for example "Milvus treats a failed original as not written and lets the retry proceed; in rare cases a write that failed with an error may still have landed, in which case the retry duplicates it." Note: this was raised in round 1 (as the claim that a failed original unconditionally releases the key), and the author qualified the wording in response at that time. The subsequent page rewrite in 5406ad0e reintroduced the unconditional "nothing landed" phrasing, so the earlier fix no longer stands and the item is re-raised. (raised by czs007, xiaocai2333)

Low

• site/en/userGuide/insert-and-delete/idempotent-requests.md:133 — "Retrying the key returns that same failed jobId forever" contradicts line 127 of the same page, which says an import key is remembered for about a day (streaming.walBroadcaster.tombstone.maxLifetime, default 24h). Once the key is evicted, a same-key retry starts a fresh job, and the server comment is explicitly bounded ("for the rest of the window", services.go:1983-1985). Suggestion: change "forever" to "for as long as the key is remembered". (raised by czs007, xiaocai2333)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:125 — The page says the import key is scoped by internal collection ID "so renaming the collection does not break a retry", but does not say the retry must use the new name. Per ddl_callbacks_import.go:309-314, only a retry naming the renamed collection resolves to the original job; one still naming the old collection is resolved by the proxy first and never reaches the dedup path. Collection is renamed between the original import and the retry, and the retry still sends the old name with the same key → name resolution fails at the proxy and the retry returns a collection-not-found error instead of the original jobId. Suggestion: add that the retry must target the renamed collection. (raised by czs007, xiaocai2333)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:49 — "You get the original result back" when a key is reused for different data is only true for inserts when the payload has the same shape. When the row count or primary-key type differs, mergeDuplicateInsertResults fails and the proxy returns error 1100 ("idempotency key was reused with a different payload; the server kept the original insert result", task_insert_streaming.go:100-121). Client reuses an explicit key for an insert with a different number of rows → the proxy returns 1100 rather than the original result, and a client branching only on "original result returned" misreads every retry as an input error. Suggestion: add a clause that a reused payload with a different row count or primary-key type returns error 1100 and the original result is kept. The author revised this sentence in an earlier round; the page rewrite simplified it again and dropped the qualifier. (raised by czs007)

[czs007]

Must-fix issues introduced by this PR: 0
Merge recommendation: Mergeable as-is — this review found no must-fix issue introduced by this PR.

Rendering the review comment now.

---

Post-verification review of the new idempotent-requests page found one medium-severity snippet regression from the rewrite and two low-severity wording and cross-reference inaccuracies, all introduced by this PR.

Medium

• site/en/userGuide/insert-and-delete/idempotent-requests.md:120 — The bulk import snippet annotates the return value as # {"jobId": "<job-id>"}, implying resp["jobId"] works, but pymilvus bulk_import() (bulk_writer/bulk_import.py:109-128) returns a requests.Response. Failure: a reader copies the snippet and reads the job id as resp["jobId"] or resp.get("jobId") → Python raises TypeError: 'Response' object is not subscriptable (or AttributeError for .get); the actual id lives at resp.json()["data"]["jobId"], as data-import/import-data.md:63 and this PR's earlier revision (commit 4556ed6) show. Suggestion: restore the pre-rewrite annotation, e.g. # resp.json()["data"]["jobId"] == "<job-id>", so the comment matches the real return shape. (raised by tinswzy)

Low

• site/en/userGuide/insert-and-delete/idempotent-requests.md:140 — "See the system configuration reference for the full list and defaults" is a dead end: reference/sys_config/configure_streaming.md has no streaming.idempotency.* entry, and both IdempotencyMaxKeyLength and WALBroadcasterTombstoneMaxLifetime are Export: false upstream (component_param.go:8753, :8766), so regeneration will not add them. Relatedly, configure_datacoord.md:1250 still lists dataCoord.import.taskRetention as 10800, contradicting the 48-hour default stated at line 134 until that reference is regenerated for the release carrying milvus #52544. Suggestion: either drop the pointer and list the relevant parameters and defaults inline on this page, or point to the specific taskRetention entry and add a note that the reference page reflects the pre-#52544 default until regenerated. (raised by tinswzy)

• site/en/userGuide/insert-and-delete/idempotent-requests.md:133 — "Retrying the key returns that same failed jobId forever" overstates the window. The failed job id is only returned while the key is still remembered, i.e. within the tombstone retention window (idempotency_index.go:48-51, tombstone_scheduler.go:110-125, default 24h), which also contradicts this page's own line 127 ("Milvus remembers an import key for about a day"). The reader's action (switch to a new key) is unchanged, so impact is minimal. Suggestion: revert to the earlier phrasing from commit 4556ed6, "for the rest of the window", or similar. (raised by czs007)

[tinswzy]

I checked every technical claim on this page against the server code and they all hold: the
two switches
and the 1100 on sending a key too early; 256 bytes of printable ASCII; the Go SDK's Upsert
rejecting a key; WithIdempotencyKey existing on both the column- and row-based options; no
time limit on the insert window; DDL forgetting a shard's keys; a retry waiting on an
in-flight original; the ~24h import key window (tombstone.maxLifetime) and the 48h
taskRetention; heavy DDL or import traffic shortening the former (tombstone.maxCount);
and none of the parameters appearing in the default milvus.yaml (they are Export: false).

I also agree with the cut from 2283 to 986 words and would not add the detail back. One
exception below.

Must fix

"Two operations honor the key today: insert and bulk import." Insert does not, yet —
milvus-io/milvus#50007 is still open, and this page targets v3.0.x. Until that ships, a
reader who follows the Insert section gets error 1100 either way: the switches do not exist
to turn on, and sending a key without them is rejected. The PR description says this
("Insert honors it once #50007 lands") but the page does not. Either gate the page on that
release or say plainly which part is not available yet.

Worth fixing

"Each shard remembers recent keys up to a byte budget." There is also a cap on how many
batches are retained (streaming.idempotency.maxRetainedChunks, 256), and for a workload
that writes a little per batch that one binds long before the byte budget does — it exists
precisely for that case. A reader estimating how long a key survives from a byte budget will
overestimate it, sometimes by a lot. This is the one place where the missing precision
changes a reader's decision, which is why I would add half a sentence: whichever comes first,
a byte budget or a cap on retained batches, and a workload writing little per batch reaches
the second one much sooner.

"At most 256 bytes of printable ASCII." 256 is the default of
streaming.idempotency.maxKeyLength, not a fixed limit. "by default" is enough; the
Configuration section already points at the parameter family.

Optional

The SDK exception note will need updating. The page notes that the Go SDK's Upsert
rejects a key outright. milvus-io/pymilvus#3784 as it stands does the opposite — it accepts
idempotency_key on upsert and delete and sends it, and the server silently ignores it.
Once that contract is settled, this note should cover both SDKs; a reader switching languages
currently gets different behaviour from the same call.