Getting set up with MinIO as S3 storage

[dagsir[bot]]

Does anyone have experience getting Dagster to work locally on a Mac with MinIO as their S3 storage?

I have MinIO <https://min.io/download#/macos|installed with brew>, have a bucket named dagster setup and a user with readwrite privileges. I have written a Python script using boto3 with the url and credentials to verify that I can connect and list the buckets successfully.

What I'm struggling with is to get Dagster to work properly. I have the username/password in a .env file and I can use Dagster logger to show that the username/password are being set correctly using os.getenv but every time I try to materialize an asset that uses the S3 IOManager I get botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied.

I have the resource setup as follows in init__.py

    service_name="s3",
    use_unsigned_session=True,
    endpoint_url="<http://localhost:9000/>",
    use_ssl=False,
)
…
…
"s3_io_manager": S3PickleIOManager(
        s3_resource=my_s3_resource,
        s3_bucket="dagster",
    ),```
Any assistance in getting the configuration setup would be appreciated.

---

*The question was originally asked in [Dagster Slack](https://dagster.slack.com/archives/C01U954MEER/p1699302331612609?thread_ts=1699302331.612609&cid=C01U954MEER).*

[bplexico]

In order to get things working I set a region value (us-west-1) in Minio and added that to the config in Dagster.

I then set the bucket to public to allow anonymous read/write to anyone since I'm testing on my laptop. Obviously, this isn't recommended for production.