Releases: NVIDIA/multi-storage-client
0.36.0
New Features
- Improve Rust client error handling to include HTTP status codes in
RustClientError. - Refactor Rust client to use
offsetandsizeinstead ofstartandendfor consistency with Python.
Bug Fixes
- Fix object listing behavior when using file or subdirectory paths.
- Make
access_timeoptional when parsing AIStore metadata to prevent errors.
0.35.0
Breaking Changes
- Add MSC prefix to progress bar suppression environment variable (
SUPPRESS_PROGRESS_BAR→MSC_SUPPRESS_PROGRESS_BAR). - Remove
msc globcommand.
New Features
This release introduces a Model Context Protocol (MCP) server for the MSC. See the documentation for more details.
- Add Model Context Protocol (MCP) server.
- Allow overwrites in
ManifestMetadataProvider. Introduce logical/physical path separation. - Make
profilesoptional in MSC configs. - Add
discard()in thePosixFileto clean up the temporary file when the atomic write is enabled. - Use
CredentialProviderinterface in Rust client. - Add Google service account credentials provider for GCS storage provider.
- Add AWS SDK/CLI external credentials process credentials provider for S3 storage providers.
Bug Fixes
- Move progress bar output from stdout to stderr.
- Parse timestamps from AIStore.
0.34.0
Breaking Changes
- Drop tracing.
New Features
This release introduces the Multi-Storage File System (MSFS), a POSIX-compatible client that enables applications to access object storage through standard POSIX file operations. While the Python Multi-Storage Client is designed for Python applications, MSFS addresses use cases where applications cannot or should not be modified, or where applications are implemented in other languages. Using FUSE, MSFS provides a POSIX access path to object storage while maintaining common mechanisms with the Python client, making it easy to adopt object storage across your entire technology stack without code changes. This initial release supports S3-compatible storage, with additional cloud providers planned for future releases. See the MSFS documentation for more details.
- Add telemetry support in MSFS.
- Add AWS config/credentials support to MSFS.
- Add AIStore S3 storage provider with configuration examples.
- Add file attribute support for MSC adaptors.
- Add
mruandpurgefactorparameters to cache configuration. - Use Rust client for multipart uploads of large BytesIO objects for better performance.
- Implement proactive credential refresh in Rust client.
- Optimize
msc syncby avoiding a temporary file when syncing to/from POSIX storage provider. - Add
source_filesparameter tosync_from()method to allow syncing a specific list of files. - Ignore hidden files by default in
msc syncoperations. - Add Etag support in HuggingFace storage provider.
Bug Fixes
- Fix List semantics in HuggingFace provider.
- Fix incorrect results from
storage_client.list()whenbase_pathcontains both bucket and prefix. - Pass timeout configurations from Python to Rust for consistent behavior.
0.33.0
- Reinitialize locks and caches when process forking to prevent deadlocks.
- Add
skip_signatureoption to access public GCS buckets without authentication. - Add
follow_symlinksoption tosync_fromoperations for symbolic link handling. - Add Parquet format support for manifest parts.
- Add
mountandumountcommands for MSC POSIX filesystem operations. - Create temporary files in cache directory to fix cross-device rename errors.
- Make boto3 connection pool size scale with CPU count for better performance.
- Make config dict immutable to fix pickling issues.
- Improve
msc lsoutput format for better readability. - Remove queue size limit when using threading for better throughput.
0.32.0
- Breaking change: Make beta telemetry initialization lazy.
- Breaking change: Drop legacy metrics.
- Cap progress bars to 99.9% if not complete.
- Support include and exclude filters in
msc sync. - Add metrics collection to
PosixFileI/O operations. - Support syncing metadata in sync operations.
- Use correct delay value for retries.
- Infer content types for the SwiftStack storage provider.
- Add
skip_signaturesupport in Rust Client for public bucket access. - Handle missing files gracefully for Zarr.
- Convert Rust buffer to Python bytes in
RemoteFileReader.
0.31.0
- Add
msc config validatecommand. - Added partial file caching with configurable chunk sizes, prefetch control, and source version validation for improved performance on large files.
- Add back support for Python 3.9 (end of life starting October 2025).
0.30.0
- Fix progress bar update during deletion.
- Improve configuration loading documentation.
0.29.0
- Drop support for Python 3.9 (end of life starting October 2025).
- Add
list_recursivemethod to the Rust client. - Rename
upload_multiparttoupload_multipart_from_filein the Rust client. - Add
upload_multipart_from_bytesmethod to the Rust client. - Rename
download_multiparttodownload_multipart_to_filein the Rust client. - Add
download_multipart_to_bytesmethod to the Rust client. - Add explicit chunk size and concurrency parameters to
upload_multipart_from_fileanddownload_multipart_to_filein the Rust client. - Accept config file paths in
StorageClientConfig.from_file(). - Limit Rust client concurrency and add retry for connection errors.
0.28.0
- Skip telemetry provider automatic initialization when unpickling to prevent multiprocessing issues.
- Fix progress bar total update when enqueueing delete tasks during sync operations.
- Optimize process and thread allocation in
msc syncwhen Rust client is enabled for better performance.
0.27.0
- Add support for refreshable credentials in the Rust client.
- Improve sync functionality with better progress tracking, stability, and error handling.
- Add Rust client user guide for better developer experience.