Add Zstd compression to the websocket messages

[ResuBaka]

Description of Changes

This add the option to the websocket/subscribe endpoint so you can use Zstd as the compression in addition to None/Brotli/Gzip.

API and ABI breaking changes

None

Expected complexity level and risk

1

Testing

• I have tested it with the simple chat example where I have enabled the zstd compression and added some logging to see if it was used
• Testing for the compression speed to find the best level

Additional

What we could look into is to use zstd dictionary features to improve the performance even more as it could possible help with the base structure of each message. The only thing that would then needed to be done is have an extra option in the enum as an ZstdDict as the client and server would need to know about the dictionary.

[CLAassistant]

All committers have signed the CLA.

[gefjon]

What's the motivation for this change? Is there some environment you want to connect to SpacetimeDB from where Zstd compression is available, but GZip and Brotli are not?

[ResuBaka]

The motivation is that in Zstd should be in general be faster when it comes to decompression. So it should at the least be better for clients.

For server it could cause a small amount of more cpu usages.

So I wanted to add it to have one more options for compression, which could depending on how big responses are be better then the other options. But to check it there would be an benchmark be needed, to see how general the speed differs from each compression option.

Do you guys currently have an easy way to benchmark mixed message compression/decompression speed?

[bfops]

Hey @ResuBaka, sorry for the delay here. We've had a lot going on.

I don't believe we have any benchmarks you could easily run to compare compression/decompression speeds. We did internally run manual benchmarks on several algos, including Zstd, brotli, and gzip, and came to the conclusion that Brotli and Gzip were best to balance server and client load. Part of the reason is that a small amount of increased server load becomes a big amount when the server is managing lots of clients.

Were you seeing issues with compression performance for your use case, or just trying to be prepared?

[cloutiertyler]

Closing as stale since we haven't had any reply here.

[clockwork-labs-bot]

Closing per the close-stale-pr-create-issue label. This work is now tracked in #4797 for reimplementation on a fresh branch.

Replacement issue: #4797

[paddor]

Commenting here since there are actual humans here. Might be of interest: zrip