Feature request: Periodic flushing / auto-batching #2302
Description
Feature Request (branch)
Hi team, have you considered periodic-flushing/auto-batching as a middle ground between auto and manual flushing?
I'm aware of the @Batch annotation for the command interface, but I don't believe there is any way to avoid command timeouts if the batch size isn't reached.
I've been looking at the case where many threads make a small number of requests (~10-20) to Redis each before flushing. In this case, there is still significant traffic to Redis in aggregate, but a connection per thread would introduces a lot of overhead, and I believe even pooling is suboptimal due to the batch size.
In a test application with 50 threads, I see batch sizes of 0-30 manually flushing across multiple threads, compared to 300-500 when flushing periodically at 1ms intervals.
Is your feature request related to a problem? Please describe
This is related to the previous issue I opened (#2289) regarding Redis CPU usage with auto-flush enabled.
In our case, we have a large request handling pool, but each thread only issues ~10 requests to Redis before flushing. We found that with auto-flush enabled, each command is written out to the socket individually, and our Redis node was seeing very high CPU usage, which we presume is from reading in commands.
Our current solution is to use a shared connection and manually flush in each thread, which returns CPU usage to what we expect in our node.
Describe the solution you'd like
- Commands are written to the command buffer
- A task running on the event loop periodically flushes the buffer at a configurable interval
- A max size can be configured, that will immediately flush the buffer.
Advantages:
- Can increase the batch size significantly compared to manually flushing with a shared connection.
- CPU usage is noticeably lower in Redis
Drawbacks:
- Inherently adds latency
- The optimal flush interval is very constrained. In my testing,
<=5ms(realistically1-5ms) proved optimal and had QPS between auto and manual flushing.
Describe alternatives you've considered
Manually flushing a shared connection remains an effective option in lieu of this.
My expected use case is when the number of commands issued between flushing for an individual thread is low, but the number of threads sharing the connection is high.
Teachability, Documentation, Adoption, Migration Strategy
If you can, explain how users will be able to use this and possibly write out a version the docs.
Maybe a screenshot or design?
A possible interface:
/**
* Disable or enable auto-batch behavior. Default is {@code false}. If autoBatchCommands is enabled,
* multiple commands can be buffered before being automatically flushed to the transport.
* Commands are buffered until the configured batch size or maximum delay is reached.
*
* @param autoBatch state of autoBatch.
*/
void setAutoBatchCommands(boolean autoBatch);
/**
* Sets the auto-batch delay. Default is {@code 5 milliseconds}. If autoBatchCommands is enabled,
* this is the maximum amount of time that will pass before buffered commands are flushed to the transport.
*
* @param delay maximum delay to auto-batch.
*/
void setAutoBatchDelay(Duration delay);
/**
* Sets the auto-batch size. Default is {@code 50}. If autoBatchCommands is enabled,
* a flush will be triggered when this number of commands have been buffered,
* regardless of how long it has been since the last flush.
*
* @param size maximum size of a batch.
*/
void setAutoBatchSize(int size);