Skip to content

Releases: NVIDIA/multi-storage-client

0.36.0

26 Nov 19:57
59780d8

Choose a tag to compare

New Features

  • Improve Rust client error handling to include HTTP status codes in RustClientError.
  • Refactor Rust client to use offset and size instead of start and end for consistency with Python.

Bug Fixes

  • Fix object listing behavior when using file or subdirectory paths.
  • Make access_time optional when parsing AIStore metadata to prevent errors.

0.35.0

20 Nov 19:19

Choose a tag to compare

Breaking Changes

  • Add MSC prefix to progress bar suppression environment variable (SUPPRESS_PROGRESS_BARMSC_SUPPRESS_PROGRESS_BAR).
  • Remove msc glob command.

New Features

This release introduces a Model Context Protocol (MCP) server for the MSC. See the documentation for more details.

  • Add Model Context Protocol (MCP) server.
  • Allow overwrites in ManifestMetadataProvider. Introduce logical/physical path separation.
  • Make profiles optional in MSC configs.
  • Add discard() in the PosixFile to clean up the temporary file when the atomic write is enabled.
  • Use CredentialProvider interface in Rust client.
  • Add Google service account credentials provider for GCS storage provider.
  • Add AWS SDK/CLI external credentials process credentials provider for S3 storage providers.

Bug Fixes

  • Move progress bar output from stdout to stderr.
  • Parse timestamps from AIStore.

0.34.0

07 Nov 23:29

Choose a tag to compare

Breaking Changes

  • Drop tracing.

New Features

This release introduces the Multi-Storage File System (MSFS), a POSIX-compatible client that enables applications to access object storage through standard POSIX file operations. While the Python Multi-Storage Client is designed for Python applications, MSFS addresses use cases where applications cannot or should not be modified, or where applications are implemented in other languages. Using FUSE, MSFS provides a POSIX access path to object storage while maintaining common mechanisms with the Python client, making it easy to adopt object storage across your entire technology stack without code changes. This initial release supports S3-compatible storage, with additional cloud providers planned for future releases. See the MSFS documentation for more details.

  • Add telemetry support in MSFS.
  • Add AWS config/credentials support to MSFS.
  • Add AIStore S3 storage provider with configuration examples.
  • Add file attribute support for MSC adaptors.
  • Add mru and purgefactor parameters to cache configuration.
  • Use Rust client for multipart uploads of large BytesIO objects for better performance.
  • Implement proactive credential refresh in Rust client.
  • Optimize msc sync by avoiding a temporary file when syncing to/from POSIX storage provider.
  • Add source_files parameter to sync_from() method to allow syncing a specific list of files.
  • Ignore hidden files by default in msc sync operations.
  • Add Etag support in HuggingFace storage provider.

Bug Fixes

  • Fix List semantics in HuggingFace provider.
  • Fix incorrect results from storage_client.list() when base_path contains both bucket and prefix.
  • Pass timeout configurations from Python to Rust for consistent behavior.

0.33.0

23 Oct 02:57
e14c821

Choose a tag to compare

  • Reinitialize locks and caches when process forking to prevent deadlocks.
  • Add skip_signature option to access public GCS buckets without authentication.
  • Add follow_symlinks option to sync_from operations for symbolic link handling.
  • Add Parquet format support for manifest parts.
  • Add mount and umount commands for MSC POSIX filesystem operations.
  • Create temporary files in cache directory to fix cross-device rename errors.
  • Make boto3 connection pool size scale with CPU count for better performance.
  • Make config dict immutable to fix pickling issues.
  • Improve msc ls output format for better readability.
  • Remove queue size limit when using threading for better throughput.

0.32.0

10 Oct 21:30

Choose a tag to compare

  • Breaking change: Make beta telemetry initialization lazy.
  • Breaking change: Drop legacy metrics.
  • Cap progress bars to 99.9% if not complete.
  • Support include and exclude filters in msc sync.
  • Add metrics collection to PosixFile I/O operations.
  • Support syncing metadata in sync operations.
  • Use correct delay value for retries.
  • Infer content types for the SwiftStack storage provider.
  • Add skip_signature support in Rust Client for public bucket access.
  • Handle missing files gracefully for Zarr.
  • Convert Rust buffer to Python bytes in RemoteFileReader.

0.31.0

30 Sep 18:46

Choose a tag to compare

  • Add msc config validate command.
  • Added partial file caching with configurable chunk sizes, prefetch control, and source version validation for improved performance on large files.
  • Add back support for Python 3.9 (end of life starting October 2025).

0.30.0

26 Sep 17:46

Choose a tag to compare

  • Fix progress bar update during deletion.
  • Improve configuration loading documentation.

0.29.0

18 Sep 20:34
06c7639

Choose a tag to compare

  • Drop support for Python 3.9 (end of life starting October 2025).
  • Add list_recursive method to the Rust client.
  • Rename upload_multipart to upload_multipart_from_file in the Rust client.
  • Add upload_multipart_from_bytes method to the Rust client.
  • Rename download_multipart to download_multipart_to_file in the Rust client.
  • Add download_multipart_to_bytes method to the Rust client.
  • Add explicit chunk size and concurrency parameters to upload_multipart_from_file and download_multipart_to_file in the Rust client.
  • Accept config file paths in StorageClientConfig.from_file().
  • Limit Rust client concurrency and add retry for connection errors.

0.28.0

28 Aug 22:43
6cfa2a7

Choose a tag to compare

  • Skip telemetry provider automatic initialization when unpickling to prevent multiprocessing issues.
  • Fix progress bar total update when enqueueing delete tasks during sync operations.
  • Optimize process and thread allocation in msc sync when Rust client is enabled for better performance.

0.27.0

22 Aug 17:06
a3d7332

Choose a tag to compare

  • Add support for refreshable credentials in the Rust client.
  • Improve sync functionality with better progress tracking, stability, and error handling.
  • Add Rust client user guide for better developer experience.