Skip to content

Commit ce370dd

Browse files
Small updates missed in MinIO support added (#675)
- Note in bucket_setup for minimum rclone version needed - use generate_s3_config whenever a s3 config is needed
1 parent e7c0c4d commit ce370dd

File tree

2 files changed

+3
-2
lines changed

2 files changed

+3
-2
lines changed

bucket_setup.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Backblaze B2 is only used as a backup storage option when the MinIO bucket is un
1717
The following will mount /silnlp on your B drive or /nlp-research on your M drive and allow you to explore, read and write.
1818
* Install WinFsp: http://www.secfs.net/winfsp/rel/ (Click the button to "Download WinFsp Installer" not the "SSHFS-Win (x64)" installer)
1919
* Download rclone from: https://rclone.org/downloads/
20+
* If you already have rclone installed, make sure it is updated to at least version 1.43 to support Backblaze
2021
* Unzip to your desktop (or some convient location).
2122
* Add the folder that contains rclone.exe to your PATH environment variable.
2223
* Take the `scripts/rclone/rclone.conf` file from this SILNLP repo and copy it to `~\AppData\Roaming\rclone` (creating folders if necessary)

silnlp/common/environment.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,7 @@ def set_resource(self, bucket_name: str, endpoint_url: str, access_key: str, sec
163163
endpoint_url=endpoint_url,
164164
aws_access_key_id=access_key,
165165
aws_secret_access_key=secret_key,
166-
config=Config(read_timeout=600),
166+
config=generate_s3_config(),
167167
# Verify is false if endpoint_url is an IP address. Aqua/Cheetah connecting to MinIO need this disabled for now.
168168
verify=False if re.match(r"https://\d+\.\d+\.\d+\.\d+", endpoint_url) else True,
169169
)
@@ -180,7 +180,7 @@ def set_s3_bucket(self):
180180
LOGGER.warning("Support for AWS S3 will soon be removed. Please set up MinIO and/or B2 credentials.")
181181
resource = boto3.resource(
182182
service_name="s3",
183-
config=Config(read_timeout=600),
183+
config=generate_s3_config(),
184184
)
185185
bucket = resource.Bucket("silnlp")
186186
register_configuration_parameter(PureS3Path("/"), resource=resource)

0 commit comments

Comments
 (0)