feat: add IO_THREAD_COUNT client property [HZ-5386]#1410
feat: add IO_THREAD_COUNT client property [HZ-5386]#1410ihsandemir wants to merge 7 commits intohazelcast:masterfrom
Conversation
Add two new client property constants to control threading: - IO_THREAD_COUNT (default 3): number of IO threads for networking - RESPONSE_THREAD_COUNT (default 2): number of response processing threads Includes static string definitions, constructor initializers, and accessor methods following the existing client_properties pattern.
When no explicit executor_pool_size is set in client_config, the user_executor_ thread pool now reads its size from the RESPONSE_THREAD_COUNT client property (default: 2), matching the Java client behavior. Previously it created an unbounded pool.
Instead of storing a single io_context& and resolver& as member fields, SocketFactory now receives them as parameters to create(). This enables callers to pass different io_context instances per connection, which is needed for multi-threaded IO support.
…ignment Replace single io_context/io_thread with vectors sized by IO_THREAD_COUNT property. New connections are assigned to IO contexts via atomic round-robin index, distributing network load across multiple threads.
| std::unique_ptr<boost::asio::io_context> io_context_; | ||
| std::vector<std::unique_ptr<boost::asio::io_context>> io_contexts_; | ||
| std::vector<std::unique_ptr<boost::asio::ip::tcp::resolver>> io_resolvers_; | ||
| std::unique_ptr<internal::socket::SocketFactory> socket_factory_; |
There was a problem hiding this comment.
This has moved (wasn't updated). Any good reason?
There was a problem hiding this comment.
no reason except that socket factory is using io_context and resolver in its methods, they are related but move is a random choice, I can revert it, no problem.
There was a problem hiding this comment.
It's not a massive problem, just wanted to check my understanding.
|
|
||
| const std::string client_properties::IO_THREAD_COUNT = | ||
| "hazelcast.client.io.thread.count"; | ||
| const std::string client_properties::IO_THREAD_COUNT_DEFAULT = "3"; |
There was a problem hiding this comment.
It would be nice if there was documentation to explain why 3 was chosen. Even if it just links to the config in the Java client or something.
There was a problem hiding this comment.
There was a study conducted a while ago reported at here which gives details about the choice of 3.
There was a problem hiding this comment.
Can we link to that in the code to help a future maintainer?
There was a problem hiding this comment.
That is internal.
There was a problem hiding this comment.
Also, doesn't cpp client verify the public configuration? What do we expect when count is meaningless?
There was a problem hiding this comment.
That is internal.
I understand that it won't help anyone outside of Hazelcast.
| for (int i = 0; i < io_thread_count; ++i) { | ||
| auto ctx = | ||
| std::unique_ptr<boost::asio::io_context>(new boost::asio::io_context); | ||
| io_guards_.push_back(std::unique_ptr<boost::asio::executor_work_guard< | ||
| boost::asio::io_context::executor_type>>( | ||
| new boost::asio::executor_work_guard< | ||
| boost::asio::io_context::executor_type>( | ||
| boost::asio::make_work_guard(*ctx)))); | ||
| io_resolvers_.push_back(std::unique_ptr<boost::asio::ip::tcp::resolver>( | ||
| new boost::asio::ip::tcp::resolver(ctx->get_executor()))); | ||
| auto raw_ctx = ctx.get(); | ||
| io_contexts_.push_back(std::move(ctx)); | ||
| io_threads_.emplace_back([raw_ctx]() { raw_ctx->run(); }); | ||
| } |
There was a problem hiding this comment.
This is definitely my inexperience, but this looks... intense.
Could it be more readable / commented?
There was a problem hiding this comment.
well, that is how C++ code looks. basically we are initializing the vectors and starting the io threads. Lock guards are used for finishing the io threads running. I will put more comments.
ihsandemir
force-pushed
the
fixIoThreadCount
branch
2 times, most recently
from
ea0b3e4 to
72351b9
Compare
ihsandemir
changed the title
3 participants
Instead of using a single IO thread, now we use similar to Java client, 3 threads. This was causing throughput to suffer where IO thread would be a bottleneck and concurrent max thread count to increase the throughput would saturate at 160 threads.
Each connection still work on one of these 3 threads.
feat: add IO_THREAD_COUNT client property
Add two new client property constant to control threading:
Includes static string definitions, constructor initializers, and
accessor methods following the existing client_properties pattern.