Tracking issues of RFC-6707: Capability Override Layer

[Xuanwo]

• RFC: RFC-6707: Capability Override Layer #6707
• First implementation: feat(core): add capability override layer #7520
• Latest scan: origin/main at a21e9b85fb

Status

The first implementation has landed. It added a shared CapabilityOverrideLayer, wired OPENDAL_TEST_CAPABILITY_OVERRIDES into the behavior test harnesses, and migrated S3 test setups for versioning, append, and write-if-match coverage.

The current service migration pass has landed for S3, OSS, COS, TOS, OneDrive, OBS, Azblob, WebDAV, HDFS, and HDFS Native. A follow-up repo-wide scan found one remaining capability-only config candidate: SFTP enable_copy only gates Capability::copy, while the copy implementation itself does not read the flag.

Implementation tasks

• Merge RFC-6707: Capability Override Layer (RFC-6707: Capability Override Layer #6707)
• Add CapabilityOverrideLayer and expose it from Rust opendal::layers (feat(core): add capability override layer #7520)
• Add shared override parsing based on explicit assignments, for example stat_with_version=false,delete_max_size=700 (feat(core): add capability override layer #7520)
• Wire OPENDAL_TEST_CAPABILITY_OVERRIDES into Rust behavior tests via testkit (feat(core): add capability override layer #7520)
• Expose the same layer/parser to Python, Node.js, Java, and .NET behavior tests (feat(core): add capability override layer #7520)
• Migrate S3 behavior test setups from service config toggles to capability overrides (feat(core): add capability override layer #7520)
• Deprecate S3 capability-only test config toggles as no-op compatibility shims (feat(core): add capability override layer #7520)

Service migration candidates

S3

• Migrate versioning capability selection to OPENDAL_TEST_CAPABILITY_OVERRIDES (feat(core): add capability override layer #7520)
• Migrate append capability selection to OPENDAL_TEST_CAPABILITY_OVERRIDES (feat(core): add capability override layer #7520)
• Migrate write-if-match capability selection to OPENDAL_TEST_CAPABILITY_OVERRIDES (feat(core): add capability override layer #7520)
• Migrate/deprecate disable_stat_with_override, which only gates stat override capabilities. (refactor(services/s3): complete capability override migration #7546)
• Migrate/deprecate delete_max_size and the deprecated batch_max_operations alias, which only tune Capability::delete_max_size. (refactor(services/s3): complete capability override migration #7546)

OSS

• Migrate/deprecate enable_versioning, which gates versioned stat/read/delete/list capabilities and the related write_with_if_not_exists capability. (refactor(services/oss): complete capability override migration #7549)
• Migrate/deprecate delete_max_size and the deprecated batch_max_operations alias, which only tune Capability::delete_max_size. (refactor(services/oss): complete capability override migration #7549)

COS

• Migrate/deprecate enable_versioning, which gates versioned stat/read/delete/list capabilities and related if-not-exists capabilities. (refactor(services/cos): complete capability override migration #7554)

TOS

• Migrate/remove enable_versioning, which gates read/delete version capabilities and write_with_if_not_exists. (refactor(services/tos): remove versioning option #7555)

OneDrive

• Migrate/deprecate enable_versioning, which gates stat/list version capabilities. (refactor(services/onedrive): deprecate versioning option #7558)

OBS

• Audit and deprecate enable_versioning: it existed in config/builder, but current capability declaration did not use it, so it is now a no-op compatibility shim. (refactor(services/obs): deprecate versioning option #7560)

Azblob

• Audit and deprecate batch_max_operations: current capability declaration uses the fixed AZBLOB_BATCH_LIMIT for delete_max_size, so the option is now a no-op compatibility shim. (refactor(services/azblob): deprecate batch option #7561)

WebDAV

• Migrate/deprecate disable_copy, which only gates the copy capability. (refactor(services/webdav): deprecate capability toggles #7562)
• Migrate/deprecate enable_user_metadata, which only gates write_with_user_metadata; namespace/prefix configuration remains separate because those values affect request construction. (refactor(services/webdav): deprecate capability toggles #7562)
• Disable write_with_user_metadata in Nextcloud and OwnCloud behavior fixtures because those endpoints do not preserve custom metadata. (fix(ci): disable WebDAV metadata test for Nextcloud and OwnCloud #7565)

HDFS and HDFS Native

• Migrate/deprecate enable_append, which gates write_can_append; HDFS Native write-path behavior now follows capability instead of a private config flag. (refactor(services/hdfs): deprecate append option #7563)

SFTP

• Migrate/deprecate enable_copy, which only gates Capability::copy; SFTP copy implementation streams data through the client and does not read this flag. The local SFTP fixture passes behavior::test_copy_file_with_ascii_name without OPENDAL_SFTP_ENABLE_COPY after enabling copy by default. (refactor(services/sftp): deprecate copy option #7568).

Not migration targets from this scan

These fields affect real request construction, credential loading, auth mode, or protocol strategy rather than only capability-based test selection, so they should not be migrated as part of RFC-6707 without a separate design:

• S3/COS/TOS/GCS credential/config loading switches such as disable_config_load and metadata-service switches.
• S3 disable_list_objects_v2, because it changes the list API version used by requests.
• S3 enable_virtual_host_style and enable_request_payer, because they affect request addressing and headers.
• Azblob sas_token based presign capabilities, because they derive from the selected auth mode.
• HF token-gated write/delete capabilities, because they derive from whether write credentials are configured.
• HTTP anonymous presign capabilities, because they derive from whether auth is configured.
• WebDAV disable_create_dir, because it changes write behavior by skipping automatic parent directory creation.
• WebHDFS disable_list_batch, because it changes the listing strategy.
• WebHDFS atomic_write_dir, because it enables multi-write by changing the temporary write path strategy.
• WebDAV metadata namespace options, because they affect PROPFIND/PROPPATCH request payloads.

Follow-ups

• Land SFTP enable_copy migration. (refactor(services/sftp): deprecate copy option #7568)
• Decide the removal schedule for deprecated service config fields after downstream bindings have migrated.
• Add user-facing docs/examples for non-test usage once the service migrations settle.
• Do another full service-config scan before closing this tracker to catch newly added capability-only toggles.