ArrayPool trimming issue for large response messages

[AnthonyLloyd]

Hi, I'm having performance and memory allocation problems with AsyncServerStreamingCall with a large response object size.
It seems the ArrayPool is trimming heavily making things worse.

I've looked at the Grpc.Net.Client code and I don't think I get anywhere by creating my own Marshallers.

As far as I understand HttpClientCallInvoker calls GrpcCall<TRequest, TResponse> which calls StreamExtensions.ReadMessageAsync.
This method uses:

• 4096 byte array from the ArrayPool
• message size byte array from the ArrayPool
• a MemoryStream to copy to for compression (so multiple non ArrayPool arrays will be allocated)
• Gives a ReadOnlySequence to the Marshaller method with the MemoryStream buffer.

I've tried implementing gRPC over WebSockets and that seems to work better because I can use my own ArrayPool that doesn't get trimmed and can also use System.IO.Pipelines efficiently.

Possibly improving the code in ReadMessageAsync to use a custom ArrayPool and also use System.IO.Pipelines would achieve the same result.

Have you seen this ArrayPool trimming problem and do you have any ideas on what I can do?

Thanks

[AnthonyLloyd]

Looking at the ArrayPool trimming code it seems this is just when the machine is at high memory use.

StreamExtensions.ReadMessageAsync still looks to be inefficient for large messages using a MemoryStream (which will allocate many arrays for large messages) and a large byte array from the ArrayPool.

Could I help improve this code?
For non compressed it could responseStream -> Pipe -> ReadOnlySequence - actually better to just be one byte array from the ArrayPool
For compressed responseStream -> DecompressionStream -> Pipe -> ReadOnlySequence

Using Pipe with pauseWriterThreshold: 0 to use it to create a single ReadOnlySequence on the PipeReader side.

Or could not use Pipe and construct a ReadOnlySequence from ArrayPool byte arrays from the stream. Using the length as a hint of what size to make these. More code though.

[AnthonyLloyd]

Have had a go at this without using Pipe as that would require a reference to be added.