Skip to content

Releases: apache/opendal

v0.57.0

01 Jun 01:51
v0.57.0
1e0fc73

Choose a tag to compare

Breaking Changes

  • core: Operator::copy, copy_options, copy_with, and blocking copy APIs now return Metadata instead of (). Custom raw copy implementations must implement the new copier flow and return server-side completion metadata from oio::Copy::close.
  • core: RpRead now carries optional Metadata observed while opening a read operation. Out-of-tree raw services that previously populated RpRead size or range fields must return RpRead::new(metadata) when read metadata is available, or RpRead::default() otherwise.
  • core: RetryInterceptor::intercept now takes a single RetryEvent<'_> argument instead of (&Error, Duration). Update custom retry interceptors to read retry error, delay, operation, and attempt data from the event.
  • observability: HTTP metrics now include the service_operation label for service-specific request names. Update Prometheus, OpenTelemetry, fastmetrics, or custom metrics consumers that assume the previous HTTP metric label set.
  • services: S3-compatible services now use skip_signature for unsigned requests. S3, OSS, and GCS keep allow_anonymous as a deprecated alias; TOS users must migrate to skip_signature because allow_anonymous has been removed. GCS skip_signature now unconditionally bypasses signing instead of falling back on credential errors.
  • services/hf: repo_type is now required for the Hugging Face service. Set repo_type explicitly, for example model, dataset, space, or bucket, instead of relying on the previous implicit model default.
  • services/tos: TosBuilder::enable_versioning and TosConfig::enable_versioning have been removed. TOS now declares versioned stat/read/delete capabilities natively.
  • bindings/java: generated ServiceConfig now uses canonical service names. Replace ServiceConfig.Huggingface with ServiceConfig.Hf; generated scheme() values for services such as Aliyun Drive, Cloudflare KV, HDFS Native, Vercel Artifacts, Vercel Blob, and Yandex Disk now return hyphenated canonical schemes instead of underscore variants.

What's Changed

Added

  • feat(bindings/python): add behavior tests for conditional read/write by @TennyZhuang in #7449
  • feat(bindings/python): expose list_with_versions and list_with_deleted in Capability by @TennyZhuang in #7445
  • feat(bindings/python): expose if_not_exists option for copy by @TennyZhuang in #7442
  • feat(bindings/python): expose DeleteOptions version and recursive params by @TennyZhuang in #7443
  • feat(metrics): Add regression test for metrics recording by @dentiny in #7362
  • feat(bindings/nodejs): add Operator.fromUri static factory by @TennyZhuang in #7471
  • feat(bindings/nodejs): add Entry.name getter by @TennyZhuang in #7470
  • feat(bindings/nodejs): add presignDelete method and Capability getter by @TennyZhuang in #7468
  • feat(bindings/dart): add File.read and File.write with error mapping by @TennyZhuang in #7473
  • feat(services/oss): support ECS RAM role credentials by @WenyXu in #7480
  • feat(retry): record more fields in the retry interceptor by @dentiny in #7406
  • feat(metrics): add service operation to metrics label by @dentiny in #7407
  • feat(bindings/python): expose stat_with_if_modified_since, stat_with_if_unmodified_since, stat_with_version in Capability by @TennyZhuang in #7444
  • feat(oss): add operation label for metrics by @dentiny in #7489
  • feat(azure): Add operation label to azure operations by @dentiny in #7488
  • feat(gcs): add service operation in metrics by @dentiny in #7487
  • feat(core): expose metadata access for raw list entries by @ngg in #7493
  • feat(hf): auto-discover token from env and credential file by @kszucs in #7490
  • feat(services/opfs): OPFS backend for OpenDAL by @jccampagne in #7199
  • feat(services/s3): support configuring assume role duration_seconds by @YuangGao in #7495
  • feat(services/goosefs): implement opendal-service-goosefs crate backed by goosefs-sdk by @XuQianJin-Stars in #7428
  • feat(service/tos): impl tos read/write/delete by @ddupg in #7350
  • feat(services/mysql): support list by @fengys1996 in #7076
  • feat(services/s3): support configuring assume role session tags by @YuangGao in #7518
  • feat(services/cos): support copy_with_if_not_exists via x-cos-forbid-… by @mikewhb in #7526
  • feat(core): add split_to and split_off to Buffer by @kimjune01 in #7513
  • feat(bindings/java): Add musl platform support by @Xuanwo in #7087
  • feat(services/azdls): Add user defined metadata support by @YuangGao in #7528
  • feat(binding/c): extract metadata from list entry by @dentiny in #7529
  • feat(binding/golang): expose metadata on list operation by @dentiny in #7534
  • feat(core): add copier api by @Xuanwo in #7527
  • feat(services/azblob): support skip_signature by @YuangGao in #7533
  • feat(services/s3): add multipart copier by @Xuanwo in #7541
  • feat(core): add capability override layer by @Xuanwo in #7520
  • feat(services/gcs): add rewrite copier by @Xuanwo in #7545
  • feat(services/s3): Rename allow_anonymous to skip_signature by @YuangGao in #7544
  • feat(core): Add content length hints for read and copy by @leiysky in #7543
  • feat(services/tos): add list support by @ddupg in #7536
  • feat(services/oss): Rename allow_anonymous to skip_signature by @YuangGao in #7556
  • feat(services/tos): Add native copy support by @ddupg in #7553
  • feat(services/tos): Rename allow_anonymous to skip_signature by @YuangGao in #7566
  • feat(services/gcs): Rename allow_anonymous to skip_signature by @YuangGao in #7567
  • feat(services/azblob): add block copier by @Xuanwo in #7551
  • feat(services/tos): implement stat and multipart upload by @ddupg in #7552
  • feat(services/tos): add versioning support by @ddupg in #7575
  • feat(services/tos): enable shared capability by @ddupg in #7580
  • feat(services/tos): implement copier by @ddupg in #7576
  • feat(services/goosefs): bump goosefs-sdk to 0.1.2 and image to v2.1.0.1 by @XuQianJin-Stars in #7588
  • feat: add reqwest-rustls-no-provider-tls feature by @valkum in #7582
  • feat(services/cos): support cosn scheme and STS security token by @WangXiaoBao1222 in #7418
  • feat(copy): apply timeout to copy operations by @dentiny in #7594
  • feat(copy): add conditional destination copy by @dentiny in #7600
  • feat(s3): validate multipart copy part number by @dentiny in #7585
  • feat(core): return metadata from copy by @Xuanwo in #7606
  • feat: C/Go bindings support layers by @yuchanns in #7611
  • feat(core): return metadata from read by @Xuanwo in #7624
  • feat(core): complete copy_with_if_match implementation by @YuangGao in #7627
  • feat(services/hf): add download_mode option and require explicit repo_type by @kszucs in #7625

Changed

  • refactor(services/s3): complete capability override migration by @Xuanwo in #7546
  • refactor(services/oss): complete capability override migration by @Xuanwo in #7549
  • refactor(services/cos): complete capability override migration by @Xuanwo in #7554
  • refactor(services/tos): remove versioning option by @Xuanwo in #7555
  • refactor(services/onedrive): deprecate versioning option by @Xuanwo in #7558
  • refactor(services/obs): deprecate versioning option by @Xuanwo in #7560
  • refactor(services/azblob): deprecate batch option by @Xuanwo in #7561
  • refactor(services/webdav): deprecate capability toggles by @Xuanwo in #7562
  • refactor(services/hdfs): deprecate append option by @Xuanwo in #7563
  • refactor(serv...
Read more

v0.56.0

01 May 07:21
v0.56.0
0fce5ec

Choose a tag to compare

Breaking Changes

  • core: Scheme-based APIs are now fully string-based. Migrate Operator::from_map and Operator::via_map call sites to from_iter and via_iter, and stop matching on Scheme values directly.
  • core: service-builder http_client hooks and TimeoutLayer::with_speed have been removed after deprecation. Move HTTP customization to layered/client APIs and replace with_speed with with_io_timeout.
  • core: low-level test helpers moved out of opendal::raw::tests. Use opendal::tests or depend on opendal-testkit directly for out-of-tree services and integrations.
  • integrations/parquet: parquet_opendal now targets Arrow 58 and ships as 0.8.0. Downstream integrations pinned to older Arrow releases must upgrade together.

What's Changed

Added

  • feat(services/oss): allow users to specify endpoint addressing style by @howardlau1999 in #6504
  • feat(bindings/ruby): publish ruby binding by @erickguan in #6539
  • feat(services/s3): add support for HTTP 429 TooManyRequests for S3-compatible services by @justinjoseph89 in #6589
  • feat: Add Operator Registry by @Xuanwo in #6608
  • feat: Add from_uri support for more object storage services by @Xuanwo in #6665
  • feat(services/webdav): Implement write returns metadata by @yunchipang in #6660
  • feat: Add from_uri support for http/webdav/ftp/sftp by @Xuanwo in #6666
  • feat(services/swift): Implement write returns metadata by @yunchipang in #6669
  • feat(buffer): add chunk iterator for Buffer by @TennyZhuang in #6672
  • feat(services/dropbox): Implement write returns metadata by @yunchipang in #6673
  • feat: Add from_uri support for all services by @Xuanwo in #6668
  • chore(bindings/python): Add Justfile and format codebase by @chitralverma in #6677
  • feat: Introduce TailCutLayer by @Xuanwo in #6680
  • feat(services/gdrive): Implement write returns metadata by @yunchipang in #6683
  • feat(bindings/python): Add stubs for Exception and Capability by @chitralverma in #6690
  • feat(bindings/python): Add stubs for some more types by @chitralverma in #6703
  • feat(bindings/python)!: Add stubs for remaining types by @chitralverma in #6720
  • feat(bindings/python)!: Stubs fix for operator and version by @chitralverma in #6728
  • RFC-6707: Capability Override Layer by @Xuanwo in #6707
  • feat(bindings/python)!: Generate stubs for Operator overloads and Scheme by @chitralverma in #6729
  • feat(bindings/nodejs): add concurrent limit layer by @kingsword09 in #6739
  • feat(bindings/python): Enable ftp/hdfs_native/sftp for python by @chitralverma in #6745
  • feat: implement content-md5 for s3 by @waynexia in #6508
  • feat(services/compfs): implement IoVectoredBuf for Buffer by @meteorgan in #6762
  • feat(services/lakefs): Implement write returns metadata by @kingsword09 in #6770
  • Add connection_pool_max_size option to memcached and redis builders by @trxcllnt in #6759
  • fix:(bindings/python) Gate service-sftp on windows targets by @chitralverma in #6777
  • feat(binding/nodejs): add TimeoutLayer, LoggingLayer and ThrottleLayer in nodejs binding by @Kilerd in #6772
  • feat(hugginface): allow specifying huggingface endpoint by @kszucs in #6801
  • feat(services/github): Implement write returns metadata by @rich7420 in #6806
  • feat(services/koofr): Implement write returns metadata by @rich7420 in #6809
  • feat: Implement concurrent write for azdls by @Xuanwo in #6819
  • feat: Add simulation for delete with recursive by @Xuanwo in #6824
  • feat: Add delete with recursive support for fs by @Xuanwo in #6827
  • feat(services/huggingface): Add repo_type=space support by @AryanBagade in #6833
  • feat(services/huggingface): Enrich metadata with ETag and security scan flags by @AryanBagade in #6834
  • feat(services/azdls): Add user defined metadata support by @zhan7236 in #6842
  • feat(services/azfile): Add user defined metadata support by @zhan7236 in #6841
  • feat(services): split cacache, dbfs, dropbox, etcd, foundationdb into separate crates by @tao12345666333 in #7031
  • feat: support if_not_exists in the Memory service by @carlsverre in #7036
  • feat(services/b2): Add user defined metadata support by @zhan7236 in #6844
  • feat(services/fs): Add user defined metadata support by @zhan7236 in #7061
  • feat(services/gdrive): implement batch recursive listing for ~200x performance improvement by @mro68 in #7059
  • feat(services/fs): add file as an alias scheme for fs by @userzhy in #7075
  • feat(layers/hotpath): add HotpathLayer integration by @PsiACE in #7083
  • feat(bindings/java): Speed up the performance of OperatorInputStream by @Xuanwo in #7088
  • feat: Binding Java's createInputStream and createOutputStream to support options by @Sahil-Shadwal in #7097
  • feat(services/webdav): Add user defined metadata support by @userzhy in #7074
  • feat: introduce foyer layer by @MrCroxx in #6366
  • feat(layers/route): impl route layer as rfc 7130 by @PsiACE in #7137
  • feat(services/azdls): add recursive deletion for azdls by @meteorgan in #7142
  • feat(services/hdfs): add delete with recursive support for hdfs by @meteorgan in #7136
  • feat(bindings/go): support presign by @yuchanns in #7147
  • feat(layers/concurrent-limit): accept custom semaphore without API break by @PsiACE in #7082
  • feat(services/memcached): Support connection via unix domain socket by @zenyanle in #7112
  • feat(webdav): disable_create_dir for servers without PROPFIND by @weihanglo in #7177
  • feat(services/swift): add conditional request headers for stat and read by @benroeder in #7208
  • feat(services/swift): add bulk delete support by @benroeder in #7210
  • feat(services/swift): add SLO multipart upload for large objects by @benroeder in #7212
  • feat(services/swift): add TempURL presigned URL support by @benroeder in #7214
  • feat(services): reduce list fallback stat for local kv backends by @Xuanwo in #7218
  • feat(services/s3): Add a way to specify a default object ACL by @antoine-de in #7186
  • feat(services/tos): add volcengine TOS support by @ddupg in #7233
  • feat(services/http): Add https as alias for http scheme by @clbarnes in #7241
  • feat(bindings/dotnet): Bindings dotnet implement by @Fatorin in #7238
  • feat(integrations/parquet): upgrade arrow-rs to 58.0.0 by @jo-migo in #7250
  • feat(services/s3): Add ACL on multipart upload by @antoine-de in #7255
  • feat(bindings/python): support options for presign operations by @atakiar in #7265
  • feat(s3): validate checksum before making HTTP requests by @dentiny in #7296
  • feat(services/sftp): add write_with_if_not_exists support by @unbrice in #7251
  • feat(services/hdfs-native): service hdfs-native support high availability (HA) cluster by @qingfeng-occ in #7248
  • feat(bindings/dotnet): standardize naming to OpenDAL and enforce RC releases by @Fatorin in #7355
  • feat(services/vercel): Add additional options to Vercel cache backend by @mmastrac in #7334
  • feat(hf): support writing and reading from both http and xet by @kszucs in #7185
  • feat(services/azfile): return write metadata by @spereyra-dev in #7381
  • feat: get_ranges 2x performance degradation by @comphead in https://github.com/apache/o...
Read more

v0.55.0

24 Nov 09:31
v0.55.0
48c48b1

Choose a tag to compare

Rust Core Upgrade to v0.55

Public API

Timestamp types now come from jiff

All public metadata APIs that previously exposed chrono::DateTime<Utc> now use jiff::Timestamp. For example, Metadata::last_modified() and related setters return/accept Timestamp values (core/src/types/metadata.rs). Update downstream crates to depend on jiff if they manipulate these timestamps or convert them to other formats.

Scheme handling is string-based

OperatorInfo::scheme() now returns &'static str instead of Scheme, and Operator::via_iter accepts impl AsRef<str> (typically the services::*_SCHEME constants). Additionally, the deprecated constructors Operator::from_map and Operator::via_map have been removed. Migrate any code that relied on the enum variants or the removed constructors to the new string-based constants and from_iter/via_iter.

List APIs only support versions

OpList::with_version()/version() and Capability::list_with_version have been removed after a long deprecation cycle. Use with_versions()/versions() on OpList and read Capability::list_with_versions instead.

S3Builder::security_token removed

S3Builder no longer exposes the deprecated security_token() helper. Use session_token() exclusively when configuring temporary credentials.

KV-style services no longer pretend to support list

Services that never returned meaningful results for Operator::list (such as D1, FoundationDB, GridFS, Memcached, MongoDB, MySQL, Persy, PostgreSQL, Redb, Redis, SurrealDB, TiKV, etc.) now rely on the default Unsupported implementation. Those features will be implemented later.

Raw API

Deprecated KV adapters removed

The legacy opendal::raw::adapters::{kv, typed_kv} modules have been deleted. Services should directly implement Access instead of depending on the adapters. Remove the corresponding imports and shim layers from any out-of-tree services.

Bindings Python Upgrade to v0.47

Breaking change: Module exports are explicit

opendal.__init__ now only re-exports the capability, exceptions, file, layers, services, types, Operator, and AsyncOperator symbols. Imports such as:

from opendal import Metadata, Layer

no longer work. Update them to use the dedicated submodules:

from opendal.types import Metadata
from opendal.layers import Layer

The legacy helper module opendal.__base has also been removed together with _Base.

Breaking change: Capability accessors renamed

Both Operator.full_capability() and AsyncOperator.full_capability() have been renamed to capability(). Adjust your code accordingly:

-caps = op.full_capability()
+caps = op.capability()

Breaking change: Service identifiers now have typed enums

The constructors for Operator / AsyncOperator provide overloads that accept opendal.services.Scheme members. While plain strings are still accepted at runtime, type checkers (pyright/mypy) expect the new enum values. Migrate code bases that relied on importing the old Scheme enum from opendal to from opendal import services and use services.Scheme.<NAME>.

What's Changed

Added

  • feat(services/oss): allow users to specify endpoint addressing style by @howardlau1999 in #6504
  • feat(bindings/ruby): publish ruby binding by @erickguan in #6539
  • feat(services/s3): add support for HTTP 429 TooManyRequests for S3-compatible services by @justinjoseph89 in #6589
  • feat: Add Operator Registry by @Xuanwo in #6608
  • feat: Add from_uri support for more object storage services by @Xuanwo in #6665
  • feat(services/webdav): Implement write returns metadata by @yunchipang in #6660
  • feat: Add from_uri support for http/webdav/ftp/sftp by @Xuanwo in #6666
  • feat(services/swift): Implement write returns metadata by @yunchipang in #6669
  • feat(buffer): add chunk iterator for Buffer by @TennyZhuang in #6672
  • feat(services/dropbox): Implement write returns metadata by @yunchipang in #6673
  • feat: Add from_uri support for all services by @Xuanwo in #6668
  • chore(bindings/python): Add Justfile and format codebase by @chitralverma in #6677
  • feat: Introduce TailCutLayer by @Xuanwo in #6680
  • feat(services/gdrive): Implement write returns metadata by @yunchipang in #6683
  • feat(bindings/python): Add stubs for Exception and Capability by @chitralverma in #6690
  • feat(bindings/python): Add stubs for some more types by @chitralverma in #6703
  • feat(bindings/python)!: Add stubs for remaining types by @chitralverma in #6720
  • feat(bindings/python)!: Stubs fix for operator and version by @chitralverma in #6728
  • RFC-6707: Capability Override Layer by @Xuanwo in #6707
  • feat(bindings/python)!: Generate stubs for Operator overloads and Scheme by @chitralverma in #6729
  • feat(bindings/nodejs): add concurrent limit layer by @kingsword09 in #6739
  • feat(bindings/python): Enable ftp/hdfs_native/sftp for python by @chitralverma in #6745
  • feat: implement content-md5 for s3 by @waynexia in #6508
  • feat(services/compfs): implement IoVectoredBuf for Buffer by @meteorgan in #6762
  • feat(services/lakefs): Implement write returns metadata by @kingsword09 in #6770
  • Add connection_pool_max_size option to memcached and redis builders by @trxcllnt in #6759
  • fix:(bindings/python) Gate service-sftp on windows targets by @chitralverma in #6777
  • feat(binding/nodejs): add TimeoutLayer, LoggingLayer and ThrottleLayer in nodejs binding by @Kilerd in #6772

Changed

  • refactor: Migrate cloudflare_kv from adapter::kv to Access instead by @kingsword09 in #6340
  • refactor: Centralize date/time handling with Timestamp wrapper by @dami0806 in #6650
  • refactor: Move Config related back to config.rs instead by @Xuanwo in #6667
  • refactor: Remove deprecated project virtiofs by @Xuanwo in #6682
  • refactor(core): remove conditional compilation in PageList trait by @crwen in #6715
  • refactor: migrate tikv service from adapter::kv to impl Access directly by @koushiro in #6713
  • refactor: migrate memcached service from adapter::kv to impl Access directly by @koushiro in #6714
  • refactor: migrate mysql service from adapter::kv to impl Access directly by @koushiro in #6716
  • refactor: migrate postgresql service from adapter::kv to impl Access directly by @koushiro in #6718
  • refactor: migrate persy service from adapter::kv to impl Access directly by @koushiro in #6721
  • refactor: migrate foundationdb service from adapter::kv to impl Access directly by @koushiro in #6722
  • refactor: migrate surrealdb service from adapter::kv to impl Access directly by @koushiro in #6723
  • refactor: migrate mongodb service from adapter::kv to impl Access directly by @koushiro in #6724
  • refactor: migrate d1 service from adapter::kv to impl Access directly by @koushiro in #6725
  • refactor: migrate gridfs service from adapter::kv to impl Access directly by @koushiro in #6726
  • refactor: Migrate sqlite from adapter::kv to Access instead by @NoxTav in #6328
  • refactor: migrate sled service from adapter::kv to impl Access directly by @koushiro in #6731
  • refactor: migrate rocksdb service from adapter::kv to impl Access directly by @koushiro in #6732
  • refactor: migrate redb service from adapter::kv to impl Access directly by @koushiro in #6733
  • refactor: Remove deprecated kv and typed_kv adapters by @kingsword09 in #6734
  • refactor(hdfs): restructure HdfsBackend and introduce HdfsCore by @kingsword09 in #6736
  • refactor(hdfs-native): restructure HdfsNativeBackend and introduce HdfsNativeCore by @kingsword09 in #6737
  • refactor(services): impl Debug for all service builders by @koushiro in #6756
  • refactor(types)!: use string-based scheme over enum-based approach by @koushiro in #6765

Fixed

  • fix(services/cloudflare_kv): Use DEFAULT_SCHEME constant for C...
Read more

v0.54.1

11 Sep 15:18
v0.54.1
b93c3ae

Choose a tag to compare

What's Changed

Added

  • feat: Conditional reader for cos by @pmupkin in #6424
  • feat: Add project layout for openval by @Xuanwo in #6427
  • feat: Add if-not-exists support for copy with azblob as example by @Xuanwo in #6447
  • feat(bindings/cpp): enhance Metadata API with comprehensive feature a… by @JackDrogon in #6445
  • feat: Add prefetch support for concurrent read by @Xuanwo in #6449
  • feat(bindings/cpp): adopt Google C++ Style Guide for API naming by @JackDrogon in #6448
  • feat(object_store): Implement copy, rename and return metadata after writing by @codephage2020 in #6452
  • feat(object_store): Implement copy_if_not_exists, put_multipart_opts and metadata in GetResult by @codephage2020 in #6468
  • chore(deps): add groups for nodejs by @asukaminato0721 in #6469
  • feat(bindings/ocaml): add more blocking api for ocaml by @asukaminato0721 in #6238
  • fix(bindings/cpp): Add capability to fix failed tests by @asukaminato0721 in #6488
  • feat(bindings/cpp): enable user-configurable features for storage services by @KinomotoMio in #6503
  • feat(services/oss): add support for security token for Aliyun OSS by @cavivie in #6511
  • feat(services/azblob): Add compatibility aliases for Apache Arrow object_store by @jackye1995 in #6527
  • feat(services/gcs): add configuration aliases for better Arrow object_store compatibility by @jackye1995 in #6526
  • feat(services/s3): add configuration aliases for better Arrow object_store compatibility by @jackye1995 in #6524
  • feat: allow using object_store as opendal's backend by @flaneur2020 in #6283
  • feat(services/opfs): Add create_dir support for OPFS by @crwen in #6542
  • feat(bindings/ruby): support file open options by @erickguan in #6538

Changed

  • refactor(website): remove unused dependencies and update node engine requirement by @kingsword09 in #6421
  • refacrtor: Reduce usage of scheme inside raw by @Xuanwo in #6457
  • refactor: Migrate etcd service to implement Access directly by @kingsword09 in #6420

Fixed

  • fix(bin/oli): Fix tests when TMPDIR is not /tmp by @XYenon in #6429
  • fix: patch futures buffer concurrent by @tisonkun in #6442
  • fix(services/oss): unnecessary header 'content-type' are included in the get_object request by @honsunrise in #6455
  • fix(services/moka): prevent the weigher from being overwritten by @koushiro in #6498
  • fix(services/azblob): Fix azblob base64 validation by @kenwoodjw in #6500
  • fix(bindings/nodejs): use value.prefetch instead of value.concurrent for prefetch option by @kingsword09 in #6502
  • fix(integrations/object_store): remove redundant into_iter() call by @KinomotoMio in #6509
  • fix(bindings/cpp): correct cxx_async namespace attribute by @kingsword09 in #6514
  • Fix: make cpp binding build robust by @devillove084 in #6460
  • fix: Do not trim trailing slash of name_node by @wForget in #6556

Docs

  • docs(release): fix CI workflow links for bindings by @kingsword09 in #6437
  • docs: Update RFC-5444 Operator From Uri by @Xuanwo in #6453
  • docs(core): update docs of building a storage service by @crwen in #6554

CI

Chore

  • chore(deps): bump quick-xml from 0.37.5 to 0.38.0 in /bin/oay by @dependabot[bot] in #6416
  • chore(deps): bump the http-serialization-utils group in /core with 6 updates by @dependabot[bot] in #6414
  • chore: Fix clippy for object_store by @Xuanwo in #6422
  • chore(deps): bump the others group across 1 directory with 17 updates by @dependabot[bot] in #6432
  • chore(website): upgrade prism-react-renderer to v2.4.1 by @kingsword09 in #6433
  • chore(deps): bump tokio from 1.46.1 to 1.47.0 in /bin/oay by @dependabot[bot] in #6466
  • chore(deps): bump tokio from 1.46.1 to 1.47.0 in /bin/oli by @dependabot[bot] in #6464
  • chore(deps): bump tokio from 1.46.1 to 1.47.0 in /bin/ofs by @dependabot[bot] in #6463
  • chore(deps): bump the others group in /core with 2 updates by @dependabot[bot] in #6478
  • chore(deps): bump indicatif from 0.17.11 to 0.18.0 in /bin/oli by @dependabot[bot] in #6474
  • chore(deps): bump the http-serialization-utils group in /core with 2 updates by @dependabot[bot] in #6476
  • chore(deps): bump tokio from 1.46.1 to 1.47.1 in /core in the async-runtime group by @dependabot[bot] in #6477
  • chore(bindings/nodejs): upgrade deps napi-rs to 3.0 by @kingsword09 in #6482
  • chore: make clippy happy again by @kingsword09 in #6493
  • chore(bindings/nodejs): update generated code comments style due to napi-derive indentation fix by @kingsword09 in #6501
  • feat: expose APIs for ErrorStatus by @tisonkun in #6513
  • chore(layers/fastmetrics): upgrade fastmetrics dependency to v0.4 by @koushiro in #6534
  • chore(deps-dev): bump crates-llms-txt from 0.0.8 to 0.1.1 in /website by @dependabot[bot] in #6548
  • chore(deps): bump logforth from 0.26.1 to 0.27.0 in /bin/oay by @dependabot[bot] in #6546
  • chore(deps): bump tempfile from 3.20.0 to 3.21.0 in /bin/oli by @dependabot[bot] in #6544
  • chore(deps): update datafusion requirement from 48.0.0 to 49.0.2 in /integrations/object_store by @dependabot[bot] in #6545
  • chore(deps): bump logforth from 0.26.1 to 0.27.0 in /bin/ofs by @dependabot[bot] in #6547
  • Bump version to 0.54.1 by @Xuanwo in #6558

New Contributors

Full Changelog: v0.54.0...v0.54.1

v0.54.0

17 Jul 07:06
v0.54.0
a1b8109

Choose a tag to compare

Upgrade Notes

Rust Core Public API

RFC-6189: Remove Native Blocking Support

OpenDAL v0.54 implements RFC-6189, which removes all native blocking support in favor of using block_on from async runtimes.

The following breaking changes have been made:

  • blocking::Operator can no longer be used within async contexts
  • Using blocking APIs now requires an async runtime
  • All Blocking* types have been moved to the opendal::blocking module

To migrate:

- use opendal::BlockingOperator;
+ use opendal::blocking::Operator;

RFC-6213: Options Based API

OpenDAL v0.54 implements RFC-6213, which introduces options-based APIs for more structured and extensible operation configuration.

New APIs added:

  • read_options(path, ReadOptions)
  • write_options(path, data, WriteOptions)
  • list_options(path, ListOptions)
  • stat_options(path, StatOptions)
  • delete_options(path, DeleteOptions)

Example usage:

// Read with options
let options = ReadOptions::new()
    .range(0..1024)
    .if_match("etag");
let data = op.read_options("path/to/file", options).await?;

// Write with options  
let options = WriteOptions::new()
    .content_type("text/plain")
    .cache_control("max-age=3600");
op.write_options("path/to/file", data, options).await?;

Remove stat_has_xxx and list_has_xxx APIs

All stat_has_* and list_has_* capability check APIs have been removed. Instead, check capabilities directly on the Capability struct:

- if op.info().full_capability().stat_has_content_length() {
+ if op.info().full_capability().stat.content_length {
    // ...
}

Fix with_user_metadata signature

The signature of with_user_metadata has been changed. Please update your code accordingly if you use this method.

Services removed due to lack of maintainer

The following services have been removed due to lack of maintainers:

  • atomicserver
  • icloud
  • nebula_graph

If you need these services, please consider maintaining them or use alternative services.

HttpClientLayer replaces update_http_client

The Operator::update_http_client() method has been replaced by HttpClientLayer:

- op.update_http_client(client);
+ op = op.layer(HttpClientLayer::new(client));

Expose presign_xxx_options API

New options-based presign APIs have been exposed:

let options = PresignOptions::new()
    .expire(Duration::from_secs(3600));
    
let url = op.presign_read_options("path/to/file", options).await?;

Rust Core Raw API

Remove native blocking support

All native blocking implementations have been removed from the raw API. Services and layers no longer need to implement blocking-specific methods.

Bindings Java Breaking changes

Removed services

The following services have been removed:

  • Chainsafe service has been removed (PR-5744) - The service has been sunset.
  • libsql service has been removed (PR-5616) - Dead service removal.

Batch operations removed

PR-5393 removes the batch concept from OpenDAL. All batch-related operations and capabilities have been removed.

Capability changes

  • write_multi_align_size capability has been removed (PR-5322)
  • Added new shared capability (PR-5328)

Options-based API

New options classes have been introduced for structured operation configuration:

  • ReadOptions - for read operations
  • WriteOptions - for write operations
  • ListOptions - for list operations
  • StatOptions - for stat operations

Example usage:

// Read with options
ReadOptions options = ReadOptions.builder()
    .range(0, 1024)
    .ifMatch("etag")
    .build();
byte[] data = operator.read("path/to/file", options);

// Write with options
WriteOptions options = WriteOptions.builder()
    .contentType("text/plain")
    .cacheControl("max-age=3600")
    .build();
operator.write("path/to/file", data, options);

Bindings Python

Breaking change: Native blocking API removed

OpenDAL has removed the native blocking API in the core. The Python binding's blocking API now uses an async runtime internally. This is a transparent change and should not affect most users, but:

  • The blocking API now requires an async runtime to be available
  • Performance characteristics may be slightly different
  • All blocking operations are now implemented as async operations running in a tokio runtime

Breaking change: Removed services

The following services have been removed due to lack of maintainers and users:

  • atomicserver - This service is no longer supported
  • icloud - This service is no longer supported
  • nebula_graph - This service is no longer supported

If you were using any of these services, you'll need to migrate to an alternative storage backend.

Breaking change: Chainsafe service removed

The Chainsafe service has been sunset and is no longer available.

What's Changed

Added

Changed

  • refactor(core/types)!: fix with_user_metadata signature by @meteorgan in #5960
  • refactor(!): Remove services lack of maintainers and users by @Xuanwo in #6263
  • refactor(services/moka)!: replace sync::Cache with future::Cache by @koushiro in #6270
  • refactor(bindings/go): Restructure FFI system with type-safe wrapper by @yuchanns in #6268
  • refactor: Migrate redis from adapter::kv to Access instead by @Xuanwo in #6291
  • refactor: Migrate moka from adapter::typed_kv to Access instead by @Xuanwo in #6300
  • refactor: Migrate memory service to implment Access directly by @Xuanwo in #6301
  • refactor: Migrate services cacache to implement Access by @Xuanwo in #6303
  • refactor!: Remove stat_has_xxx and list_has_xxx by @Xuanwo in #6313
  • re...
Read more

v0.48.3

17 Jul 06:39
8ed2692

Choose a tag to compare

refactor(website): remove unused dependencies and update node engine …

v0.53.3

27 May 06:56
v0.53.3
4e00288

Choose a tag to compare

What's Changed

Added

Changed

  • refactor(bindings/java): deprecate append in favor of write with append=true by @kingsword09 in #6169
  • refactor(core): Put Backtrace in a box to reduce error size by @Xuanwo in #6193

Fixed

  • fix(bindings/go): update dependencies by @JupiterRider in #6175
  • fix(integrations/object_store): fix double percent encoding by @Colerar in #6166
  • fix(services/fs): Make fs services work on our MSRV by @Xuanwo in #6183
  • fix(services/s3): Remove not needed check for batch delete by @Xuanwo in #6206

Docs

CI

  • ci: uses taiki-e/install-action to replace cargo insatll by @xxchan in #6168
  • ci(bindings/node): Add contents permissions for nodejs release by @Xuanwo in #6182
  • ci(bindings/java): Enable zig build for java by @Xuanwo in #6181
  • ci: Fix behavior tests for go bindings by @Xuanwo in #6199

Chore

New Contributors

Full Changelog: v0.53.2...v0.53.3

v0.53.2

13 May 09:31
v0.53.2
dfb1940

Choose a tag to compare

What's Changed

Added

Changed

  • refactor(core): Deprecate not used options in OpList by @Xuanwo in #6050
  • refactor(services/gdrive): move raw requests to core by @jorgehermo9 in #6088
  • refactor(services/ghac): move raw http calls to core by @jorgehermo9 in #6095
  • refactor(services/yandex_disk): Move raw request to core by @jorgehermo9 in #6090
  • refactor(services/azdls): Refactor raw request send in writer and bac… by @jorgehermo9 in #6109
  • refactor(services/seafile): Refactor raw request send in writer and lister by @jorgehermo9 in #6104
  • refactor(services/azblob): Refactor raw request send in writer by @jorgehermo9 in #6102
  • refactor(services/aliyun_drive): Move raw request to core by @jorgehermo9 in #6089
  • refactor(services/webhdfs): Refactor raw request send in writer and backend by @jorgehermo9 in #6113
  • refactor(bindings/go): add ffiCall type for FFI function signature by @yuchanns in #6158

Fixed

  • fix(core): Only run size tests on 64bit platforms by @cryptomilk in #6078
  • fix(bindings/go): Reader returns io.EOF at the end of file by @yuchanns in #6150
  • fix(core/services/fs): Returning empty dir while list a file path by @Xuanwo in #6154

Docs

  • doc(services/onedrive): add a comment about onedrive's writer by @erickguan in #6120

CI

  • ci(bindings/go): include sqlite service into behavior tests by @yuchanns in #6039

Chore

  • chore: delete unused code in scripts by @yihong0618 in #6084
  • chore(integrations/unftp): Make compatible with latest libunftp by @hannesdejager in #6094
  • chore(deps): bump logforth from 0.23.1 to 0.24.0 in /bin/ofs by @dependabot in #6137
  • chore(deps): bump assert_cmd from 2.0.16 to 2.0.17 in /bin/oli by @dependabot in #6138
  • chore(deps): bump logforth from 0.23.1 to 0.24.0 in /bin/oay by @dependabot in #6139
  • chore(deps): update dav-server requirement from 0.7.0 to 0.8.0 in /integrations/dav-server by @dependabot in #6133
  • chore(deps): bump golangci/golangci-lint-action from 6 to 7 by @dependabot in #6134

New Contributors

Full Changelog: v0.53.1...v0.53.2

v0.53.1

19 Apr 04:01
v0.53.1
551b361

Choose a tag to compare

What's Changed

Added

Changed

  • refactor: Remove dead code ConcurrentFutures by @Xuanwo in #5939
  • refactor(layers/tracing): Ensure the entire async function been traced by @Xuanwo in #6000

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: v0.53.0...v0.53.1

v0.53.0

05 Apr 03:29
v0.53.0
12a076c

Choose a tag to compare

Upgrade to Rust Core v0.53

Public API

Supabase service is now an S3-compatible servcice

Supabase Storage is now an S3-compatible service instead: https://github.com/supabase/storage.

We removed the supabase native service support in OpenDAL v0.53. Users who want to access Supabase Storage can use the S3 service instead.

All metrics related layers have been refactored

All metrics layers have been refactored:

  • PrometheusLayer
  • PrometheusClientLayer
  • MetricsLayer

They are now provides more metrics and more detailed information. All their public API have been redesigned.

For more details, please refer to opendal::layers::observe's module documentation.

Operator::default_executor has been replaced by Operator::executor

In opendal v0.53, we introduced a new concept of Context which is used to store the context of the current operator. Thanks to this design, we can now get and set the executor and http_client for given Operator instead.

All services http_client API has been deprecated and replaced by Operator::update_http_client API.

OpenDAL MSRV bumped to 1.80

Since v0.53, OpenDAL will require Rust 1.80.0 or later to build.

Raw API

Operation enum merge

To reduce the complexity of the Operation, we have merged the duplicated Operation.

For example:

  • Operation::ReaderRead has been merged into Operation::Read
  • Operation::BlockingRead has been merged into Operation::Read

Upgrade to Nodejs Binding v0.48

Breaking change

Public API

Now, nodejs binding op.is_exist changed to op.exists to align with nodejs API style.

What's Changed

Added

Changed

  • refactor(!): Supabase is now an S3-compatible servcices by @Xuanwo in #5663
  • refactor: Migrate s3 services to context based http client by @Xuanwo in #5676
  • refactor: Migrate oss services to context based http client by @Ziy1-Tan in #5681
  • refactor: Migrate obs services to context based http client by @Ziy1-Tan in #5682
  • refactor: Migrate cos services to context based http client by @Ziy1-Tan in #5683
  • refactor(bindings/node)!: Change is_exist to exists to align with nodejs API style by @yihong0618 in #5731
  • refactor(services/redis): Implement ConnectionLike for RedisConnection by @Xuanwo in #5748
  • refactor!: Remove opendal-compat which is not maintained by @Xuanwo in #5754
  • refactor: Migrate github services to context based http client by @miroim in #5764
  • refactor(gcs): Migrate to context based http client by @leiysky in #5778
  • refactor(core/raw)!: Merge blocking and async operations by @Xuanwo in #5789
  • refactor(core/raw)!: Use AccessorInfo instead of seperate fields by @Xuanwo in #5796
  • refactor: Migrate aliyun_drive services to context based http client by @miroim in #5815
  • refactor: Migrate azfile services to context based http client by @miroim in #5816
  • refactor: Migrate dropbox services to context based http client by @miroim in #5827
  • refactor: Migrate upyun services to context based http client by @miroim in #5829
  • refactor(core): Migrate BlockWriter to use executor from context by @Xuanwo in #5834
  • refactor(core): Migrate MultipartWrite Executor to context based by @Xuanwo in #5835
  • refactor(core): Migrate PositionWrite Executor to context based by @Xuanwo in #5836
  • refactor(core)!: Migrate to context based executor by @Xuanwo in #5838
  • refactor: tidy binding java code by @tisonkun in #5840
  • refactor: Migrate yandex_disk services to context based http client by @miroim in #5841
  • refactor: Migrate alluxio services to context based http client by @miroim in #5842
  • refactor: Migrate b2 service to context based http client by @miroim in #5843
  • refactor: migrate azblob services to context based http client by @Ziy1-Tan in #5845
  • refactor: Migrate gdrive service to context based http client by @miroim in #5861
  • refactor: Migrate pcloud service to context based http client by @miroim in #5866
  • refactor!: Bump OpenDAL MSRV to 1.80 by @Xuanwo in #5868
  • refactor: Introduce VercelArtifactsCore for improved service structure by @miroim in #5873
  • refactor: Migrate vercel_artifacts service to context based http...
Read more