new feature: allow selection of TLS backend.

[valkum]

Feature Description

We would like to stick with ring as the default crypto provider and also want to avoid building aws-lc-rs.

Problem and Solution

#7365 placed a hard dependency on aws-lc-rs by enabling the reqwest/rustls feature.

opendal should offer a reqwest/rustls-no-provider option that allows you to enable any rustls backend you want (ring, aws-lc-rs).

Additional Context

The chain goes a bit further, because reqsign-google uses a jsonwebtoken version that also has aws-lc-rs enabled. I filed apache/opendal-reqsign#754.

Are you willing to contribute to the development of this feature?

• Yes, I am willing to contribute to the development of this feature.

[Xuanwo]

Thank you for raising this! My current plan is make reqwest entirely optional so that users can plug whatever http client (and the tls backend) they want in opendal.

But the http client is so deep in opendal, so we might need to figure how to extract such a crate.

[valkum]

Are you open to PRs in the meantime? As in, an immediate fix by adding a feature? This is basically preventing us from updating a dependency that adopted a new OpenDAL version.

[Xuanwo]

Are you open to PRs in the meantime? As in, an immediate fix by adding a feature? This is basically preventing us from updating a dependency that adopted a new OpenDAL version.

Makes sense. Welcome!

[valkum]

Another intermediate option, and a first step in your direction, would be to just take in a reqwest client during setup. This also has the benefit of reducing the overall anumberof reqwest clients during a program's lifetime. It also makes custom configurations of reqwest feasible (using hickory, setting proxies, http1-3 configs). As in, if you want to configure reqwest, pass in your own reqwest. I assume opendal then still needs a fallback if the user did not provide one, but this fallback could be provided via a feature flag.
I can prepare both PRs for your consideration.

[Xuanwo]

Another intermediate option, and a first step in your direction, would be to just take in a reqwest client during setup. This also has the benefit of reducing the overall anumberof reqwest clients during a program's lifetime. It also makes custom configurations of reqwest feasible (using hickory, setting proxies, http1-3 configs). As in, if you want to configure reqwest, pass in your own reqwest. I assume opendal then still needs a fallback if the user did not provide one, but this fallback could be provided via a feature flag. I can prepare both PRs for your consideration.

Already supported by HttpClientLayer

[valkum]

Yes, just found this. But for it to be useful, in terms of preventing aws-lc-rs as a dependency, it would need a feature which disables the construction of the default HttpClient.
E.g. Gcs calls HttpClient::default() via AccessorInfo::default() here

The idea is to hide any construction of a default (or fallback) HttpClient behind a feature.