backup: probe etcd at task start and bound every etcd read

[czs007]

issue: #1216

A backup run with --backup_index_extra can hang forever on its first etcd read,
silently, after every collection has already been copied. Three things combine:
the client is created without ever dialling, the reads carry no deadline, and both
etcd steps run last, so an unreachable etcd is only discovered at the end of the
backup.

Reported on a deployment where all 69 collections and the RBAC metadata were
written, and the run then stopped dead on

start to get index info from etcd  prefix=by-dev/meta/field-index/462915446997590087/

with nothing after it in a 4.9 MB log.

What changed

core/backup/etcd.go (new): etcdMeta wraps the client with the endpoints it
was built from, because clientv3 never reports which address a call was talking to
and a bare context deadline exceeded gives an operator nothing to act on.

• probe establishes that etcd answers before the backup starts copying. One
  reachable member is enough — the client fails over, so a single down member of an
  otherwise healthy cluster must not fail a backup. An unreachable cluster reports
  every endpoint with its own error.
• getPrefix bounds the read with a deadline: 10s, matching etcd.requestTimeout,
  the knob Milvus bounds its own etcd calls with. There is no reason for this tool
  to wait longer than the server it reads from.

core/backup/task.go: the probe runs in initClients, inside the existing
if t.option.BackupIndexExtra branch — the only case that builds an etcd client at
all — so a cluster whose etcd this tool cannot reach fails in the first seconds
rather than after the data is copied.

core/backup/coll_index_extra_task.go, core/backup/coll_dyn_field_task.go:
read through etcdMeta instead of calling kv.Get with the unbounded task context.
The dynamic field task's scan is also scoped to the backed-up collections: the key
is root-coord/fields/{collectionID}/{fieldID}, so the collection id is already the
first segment, and the previous cluster-wide read fetched every field of every
collection in order to keep at most one dynamic field per backed-up collection. The
index extra scan was scoped this way in #1055; this one was not, and its size
tracked the instance rather than the backup — which is what makes a bounded read
safe here.

Why a probe rather than a blocking dial

clientv3.New does not dial unless the config carries grpc.WithBlock(), so
construction succeeds against endpoints that do not exist. Adding WithBlock would
not be enough: it proves the TCP connection, not that the peer speaks etcd, and an
address that accepts the connection and then never answers is exactly the reported
failure. WithBlock is also deprecated. An explicit Status settles both and
doubles as the start-of-task check.

Tests

core/backup/etcd_test.go covers the probe: reachable, one member down, all
unreachable naming every endpoint, a silent endpoint that must not block, and no
endpoint configured. TestEtcdMeta_ProbeWithRealClient pins the premise of the fix
with a real clientv3 client — clientv3.New against an address nothing listens on
returns no error, and only the probe reports it. The two task tests assert that an
unreachable etcd fails their Execute with the endpoint in the message, and that
each scan asks only for the collections in the backup.

Not in this change

etcd TLS. MilvusEtcdConfig has only Endpoints and RootPath, so a deployment
whose etcd requires client certificates cannot be read by this tool at all; Milvus
itself supports etcd.ssl.enabled with tlsCert / tlsKey / tlsCACert. That is
the fourth defect in #1216 and is left open, which is why this says issue rather
than fixes.

🤖 Generated with Claude Code

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: czs007
To complete the pull request process, please assign alohaha22 after the PR has been reviewed.
You can assign the PR to them by writing /assign @alohaha22 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov-commenter]

Codecov Report

❌ Patch coverage is 77.17391% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.26%. Comparing base (9b4767a) to head (beed9fa).

Files with missing lines	Patch %	Lines
core/backup/coll_dyn_field_task.go	69.69%	8 Missing and 2 partials ⚠️
core/backup/task.go	18.18%	9 Missing ⚠️
core/backup/coll_index_extra_task.go	83.33%	1 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (77.17%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project status has failed because the head coverage (47.26%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1218      +/-   ##
==========================================
+ Coverage   46.89%   47.26%   +0.36%     
==========================================
  Files         140      141       +1     
  Lines       12845    12877      +32     
==========================================
+ Hits         6024     6086      +62     
+ Misses       6386     6354      -32     
- Partials      435      437       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.