diff --git a/docs/_static/thread-pool-diagram.png b/docs/_static/thread-pool-diagram.png new file mode 100644 index 00000000000..d9312209d6e Binary files /dev/null and b/docs/_static/thread-pool-diagram.png differ diff --git a/docs/threadpool.md b/docs/threadpool.md index 186e7816dc3..01d0448fe4e 100644 --- a/docs/threadpool.md +++ b/docs/threadpool.md @@ -1,250 +1,472 @@ # Thread pool -Thread pooling can improve performance and scalability for MySQL databases. This technique reuses a fixed number of threads to handle multiple client connections and execute statements. It reduces the overhead of creating and destroying threads and avoids the contention and context switching that can occur when there are too many threads. +## Introduction -If you have fewer than 20,000 connections, using the thread pool does not provide significant benefits. It’s better to keep thread pooling disabled and use the default method. +Thread pooling improves performance and scalability for MySQL-compatible databases by +reusing a fixed number of pre-created threads to manage multiple client +sessions. This design reduces resource overhead, lowers contention, and avoids +context-switching bottlenecks during high concurrency. -The default method, called one-thread-per-connection, creates a new thread for each client that connects to the MySQL server. This thread manages all queries and responses for that connection until it’s closed. This approach works well for a moderate number of connections, but it can become inefficient as the number of connections increases. +The default MySQL method creates one thread per client connection. This approach +works efficiently under moderate connection loads. However, as the number of active +connections increases, especially beyond 20,000, system overhead becomes +significant, and throughput decreases. -MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of several thread groups, each managing client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit, and others. +Percona Server for MySQL includes an integrated thread pool that replaces the +default model. The thread pool manages connections more efficiently by queuing +work and reusing threads, especially in OLTP environments with many short-lived +queries. -The thread pool plugin consists of several thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups using the round-robin method. This method assigns threads fairly and efficiently. Here's how it works: +## Version-specific notes -1. The thread pool starts with a set number of thread groups. +Thread pool support in Percona Server for MySQL evolved across releases. Review +the following version details to avoid configuration errors and ensure +compatibility. -2. When a new task arrives, the pool needs to assign it to a group. + +### 8.0.14 -3. It does this by going through the groups in order, one by one. +Beginning with 8.0.14, the upstream implementation of `admin_port` replaced the +previous mechanism that used `extra_port` and `extra_max_connections`. + +Removed: -4. Let's say you have four thread groups. The assignment would work like this: - - Task 1 goes to Group 1 - - Task 2 goes to Group 2 - - Task 3 goes to Group 3 - - Task 4 goes to Group 4 - - Task 5 goes back to Group 1 - -5. This pattern continues, always moving to the next group and starting over when it reaches the end. +* These options are removed:`extra_port` and `extra_max_connections`. They are no longer recognized and cause server startup failure. + +Migration steps: -6. Each group handles its assigned tasks using its available threads. +* Remove all references to `extra_port` and `extra_max_connections` from the + configuration file before upgrading to 8.0.14 or later. -This round-robin approach spreads work evenly across all groups. It prevents any single group from getting overloaded while others sit idle. This method helps maintain balanced performance across the system. +* Use the `admin_port` variable to configure administrative access if needed. -MySQL executes statements using one thread per client connection. When the number of connections increases past a specific point, performance degrades. -This feature introduces a dynamic thread pool, which enables the server to maintain top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot lock contentions. The thread pool is most effective with `OLTP` workloads (relatively short CPU-bound queries). + +### 8.0.12-1 -Set the thread pool variable [thread_handling](#thread_handling) to `pool-of-threads` by adding the following line to `my.cnf`: +Percona Server for MySQL 8.0.12-1 introduced native support for the thread pool. +This version ported the feature from Percona Server 5.7. -```text -thread_handling=pool-of-threads +## Core concepts and usage examples + +### Fixed pool of threads + +The server initializes a specified number of threads during the startup process and maintains these active threads to handle incoming queries efficiently. The server minimizes the overhead associated with creating and destroying threads for each query by reusing threads, enhancing overall performance and responsiveness. + +The following configuration setting activates the thread pool model: + +```ini +[mysqld] +thread_handling = pool-of-threads +``` + +In this model, the server allocates threads from the pool to process queries as they arrive, allowing for better resource management and improved throughput, particularly in environments with high query loads. This approach ensures that available threads are quickly assigned to incoming requests, enabling timely query processing while optimizing system resources. + +### How the thread pool works + +When a client connects, the server assigns the connection to a thread group using a round-robin method. Each group consists of a listener thread and multiple worker threads. The listener thread monitors active connections and queues incoming statements for processing. + +The server assigns new connections to thread groups using a round-robin method. This approach balances workload evenly and prevents any group from becoming a bottleneck. + +Connection 1 goes to Group 1 + +Connection 2 goes to Group 2 + +Connection 3 goes to Group 3 + +Connection 4 goes to Group 4 + +Connection 5 returns to Group 1 + +New queries are routed to either the high-priority queue or the low-priority queue. A query enters the high-priority queue if the connection has an open transaction and there are available high-priority tickets, which allow certain queries to be processed more quickly, ensuring that critical operations receive immediate attention. All other queries are directed to the low-priority queue. + +Worker threads first scan the high-priority queue for queries to process. Once all high-priority queries have been handled, the threads process queries from the low-priority queue. After completing a query, a worker thread becomes idle and waits for the next task, helping to avoid excessive thread creation and maintaining steady performance under load. + +The thread pool adapts to shifts in workload by redistributing queued tasks and prioritizing available threads. This dynamic management improves system responsiveness while minimizing the number of threads in use. + +Thread pool diagram + +### Thread groups + +The thread pool organizes threads into distinct groups to manage client connections efficiently. Each group consists of a listener thread and multiple worker threads. The listener thread monitors incoming client connections and queuing requests for processing. Meanwhile, the worker threads handle the actual processing of these requests, allowing for concurrent execution of tasks. + +```ini +[mysqld] +thread_pool_size = 8 ``` -Although the default values for the thread pool should provide good performance, additional tuning should be performed with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks. +This configuration spreads work across eight thread groups. -Due to the following differences, this implementation is not compatible with upstream: +### Priority queues -* Built into the server, upstream implements the thread pool as a plugin +Each thread group maintains both a high-priority and low-priority queue to manage incoming queries effectively. When a new query arrives, the system determines its placement in one of the two queues based on the availability of high-priority tickets and the state of the transaction. The thread pool prioritizes processing by constantly checking the high-priority queue first, ensuring that critical queries receive immediate attention before any lower-priority queries are addressed. This approach optimizes resource allocation and enhances overall system responsiveness. -* Does not minimize the number of concurrent transactions -Priority Queue: +### High-priority ticketing -A queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap, or a binary search tree. It can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value. +Each connection begins with a set number of high-priority tickets that allow for the prioritization of a limited number of queries. Every time the thread pool promotes a query to the high-priority queue, one ticket is used. When queries are submitted to the server, each query is assigned a "ticket" indicating its priority level, which the thread pool uses to determine the processing order. -## Version specific information +Here’s how the ticket system works in more detail: -Starting with 8.0.14, Percona Server for MySQL uses the upstream implementation of the [`admin_port`](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_admin_port). The variables [extra_port](#extra_port) and [extra_max_connections](#extra_max_connections) are removed and not supported. Remove the `extra_port` and `extra_max_connections` variables from your configuration file before upgrading to 8.0.14 or higher. In 8.0.14 or higher, the variables cause a boot error, and the server refuses to start. +| Feature | Description | +|-----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Ticket Assignment | When a query is submitted, it is evaluated based on its priority. High-priority queries receive a special ticket that allows them to be processed ahead of lower-priority queries. This assignment ensures that critical tasks are addressed promptly. | +| Queue Management | The thread pool maintains separate queues for high-priority and low-priority queries. High-priority queries are placed in a queue that is processed first, while lower-priority queries wait in a separate queue. This separation helps manage resources effectively and ensures that important tasks do not get delayed. | +| Resource Allocation | The thread pool allocates resources based on the tickets. When a thread becomes available, it checks the high-priority queue first. If there are high-priority queries waiting, those queries are executed before any from the lower-priority queue. This approach maximizes the responsiveness of the system for critical operations. | +| Dynamic Adjustment | The ticket system allows for dynamic adjustments based on the current workload. If the number of high-priority queries increases, the system can adapt by allocating more resources to handle them efficiently, ensuring that performance remains optimal even during peak times. | -Implemented in 8.0.12-1: We ported the `Thread Pool` feature from Percona Server for MySQL 5.7. +## Configure thread pool -## Priority connection scheduling +For example, this configuration setting in the server's configuration file, `my.cnf`, specifies that each new connection receives five high-priority tickets. These tickets allow up to five queries to be processed before lower-priority ones. -The thread pool limits the number of concurrently running queries. The number of open transactions may remain high. Connections with already-started transactions are added to the end of the queue. A high number of open transactions has implications for the currently running queries. The [thread_pool_high_prio_tickets](#thread_pool_high_prio_tickets) variable controls the high-priority queue policy and assigns tickets to each new connection. +```ini +[mysqld] +thread_pool_high_prio_tickets = 5 +``` + +By executing this command in the client, users enable the server to prioritize up to five queries for processing ahead of lower-priority queries. This setting is beneficial in environments where certain operations require immediate attention. The change applies globally, affecting all new connections established after the command is executed. -The thread pool adds the connection to the high-priority queue and decrements the ticket if the connection has the following attributes: +```mysql +mysql> mysql> SET GLOBAL thread_pool_high_prio_tickets = 5; +``` -* Has an open transaction +Either of these configurations assign five tickets to each new connection. -* Has a non-zero number of high-priority tickets -Otherwise, the variable adds the connection to the low-priority queue with the initial value. +### High-priority modes -Each time, the thread pool checks the high-priority queue for the next connection. When the high-priority queue is empty, the thread pool picks connections from the low-priority queue. The default behavior is to put events from already started transactions into the high-priority queue. +The [`thread_pool_high_prio_mode`](#thread_pool_high_prio_mode) setting is an option that manages how queries are scheduled in the thread pool using a ticket system. -If the value equals `0`, all connections are put into the low-priority queue. If the value exceeds zero, each connection could be put into a high-priority queue. +When this setting is turned on, the system prioritizes high-priority queries. Adjusting this setting can improve the performance of your applications, especially during busy times. -The [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable prioritizes all statements for a connection or assigns connections to the low-priority queue. To implement this new [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable +### Configuration and monitoring -## Low-priority queue throttling +Administrators can fine-tune thread pool behavior by adjusting dynamic and static variables. -One case that can limit thread pool performance and even lead to deadlocks under high concurrency is when thread groups are oversubscribed due to active threads reaching the oversubscribe limit. Still, all/most worker threads are waiting on locks currently held by a transaction from another connection that is not currently in the thread pool. -In this case, the oversubscribe limit does not account for those threads in the pool that marked themselves inactive. As a result, the number of threads (both active and waiting) in the pool grows until it hits the [`thread_pool_max_threads`](#thread_pool_max_threads) value. If the connection executing the transaction holding the lock has managed to enter the thread pool by then, we get a large (depending on the [`thread_pool_max_threads`](#thread_pool_max_threads) value) number of concurrently running threads and, thus, suboptimal performance. Otherwise, we get a deadlock as no more threads can be created to process those transaction(s) and release the lock(s). +```ini +[mysqld] +thread_pool_stall_limit = 100 +``` -Such situations are prevented by throttling the low-priority queue when the total number of worker threads (both active and waiting ones) reaches the oversubscribe limit. If there are too many worker threads, do not start new transactions; create new threads until queued events from the already-started transactions are processed. +```mysql +mysql> SET GLOBAL thread_pool_stall_limit = 100; +``` + +This value, in milliseconds, limits how long a task can stall before being redistributed. As with other thread pool settings, existing sessions retain their previous values. + + +## Thread Pool Configuration Reference Sheet + +A quick overview of system variables available for configuring the thread pool in Percona Server for MySQL. + +

General Variables

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Variable nameDefaultScopeDynamicConfig fileDescription
thread_handlingone-thread-per-connectionGlobalNoYesChooses the thread model. Use pool-of-threads to enable thread pooling.
thread_pool_size16GlobalNoYesDefines the number of thread groups.
thread_pool_oversubscribe3GlobalYesYesDefines per-group queue saturation threshold.
+ +

Priority Handling

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Variable nameDefaultScopeDynamicConfig fileDescription
thread_pool_high_prio_tickets4294967295GlobalYesYesMaximum high-priority executions allowed per connection.
thread_pool_high_prio_modetransactionsGlobalYesYesDetermines which queries receive high priority.
+ +

Thread Management

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Variable nameDefaultScopeDynamicConfig fileDescription
thread_pool_max_threads1000GlobalNoYesMaximum number of worker threads per group.
thread_pool_stall_limit6GlobalYesYesWait time in milliseconds before reassigning stalled queries.
thread_pool_idle_timeout60GlobalYesYesSeconds before idle threads above min are released.
+ +

Extra Port Behavior

+
+ Deprecated variables (click to expand) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Variable nameDefaultScopeDynamicConfig fileDescription
extra_port0GlobalNoYesLegacy variable replaced by admin_port.
extra_max_connections1GlobalNoYesLegacy limit for extra_port access.
+
+ +

Status Metrics (Read-Only)

+ + + + + + + + + + + + + + + + + + + + + + + +
Variable nameScopeDynamicDescription
threadpool_threadsSessionN/ATotal worker threads across all groups.
thread_pool_idle_threadsSessionN/AIdle worker threads available for reuse.
+ +### Observability Commands + +Use these statements to inspect runtime thread pool behavior: + +```mysql +mysql> SHOW STATUS LIKE 'Threadpool%'; +mysql> SHOW ENGINE THREAD_POOL STATUS; +``` -## Handling long network waits +These commands reveal thread activity, idle counts, and queue saturation by group. -Specific workloads (large result sets, BLOBs, slow clients) can wait longer on network I/O (socket reads and writes). Whenever the server waits, this should be communicated to the thread pool so it can start a new query by either waking a waiting thread or sometimes creating a new one. This implementation has been ported from *MariaDB* patch MDEV-156. -## System variables +Note: For dynamic variables, changes apply only to new connections. Existing sessions retain their original settings until reconnect. + +## Configuration reference + +This section consolidates key thread pool variables, their behavior, and how to +configure them both statically and dynamically. + ### `thread_handling` -| Option | Description | -|--------------|---------------------------| -| Command-line | Yes | -| Config file | Yes | -| Scope | Global | -| Dynamic | No | -| Data type | String | -| Default | one-thread-per-connection | +Controls which thread management model the server uses. +* Allowed values: `one-thread-per-connection`, `pool-of-threads` -This variable defines how the server handles threads for connections from the client. +* Description: Use `pool-of-threads` to enable the thread pool. This setting must + be configured in `my.cnf` before server startup. -| Values | Description | -|---------------------------|--------------------------------------------------------| -| one-thread-per-connection | One thread handles all requests for a connection | -| pool-of-threads | A thread pool handles requests for all connections | -| no-threads | A single thread for all connections for debugging mode | +* Default: `one-thread-per-connection` -### `thread_pool_idle_timeout` +* This value is not dynamic and requires a server restart. -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global | -| Dynamic: | Yes | -| Data type: | Numeric | -| Default value: | 60 (seconds) | -This variable can limit the time an idle thread should wait before exiting. +### `thread_pool_size` -### `thread_pool_high_prio_mode` +Defines the number of thread groups. + +* Allowed values: Any positive integer (commonly 2 to 64) -This variable provides more fine-grained control over high-priority scheduling globally or per connection. +* Description: Each group manages client connections with its own listener and + worker threads. -The following values are allowed: +* Default: 16 -* `transactions` (the default). In this mode, only statements from already started transactions may go into the high-priority queue depending on the number of high-priority tickets currently available in a connection (see thread_pool_high_prio_tickets). +* This value is static and requires a server restart. -* `statements`. In this mode, all individual statements go into the high-priority queue, regardless of the transactional state and the number of available high-priority tickets. Use this value to prioritize `AUTOCOMMIT` transactions or other statements, such as administrative ones. Setting this value globally essentially disables high-priority scheduling. All connections use the high-priority queue. -* `none`. This mode disables the priority queue for a connection. Certain types of connections, such as monitoring, are insensitive to execution latency and do not allocate the server resources that would impact the performance of other connections. These types of connections do not require high-priority scheduling. Setting this value globally essentially disables high-priority scheduling. All connections use the low-priority queue. ### `thread_pool_high_prio_tickets` -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global, Session | -| Dynamic: | Yes | -| Data type: | Numeric | -| Default value: | 4294967295 | +Limits the number of high-priority queries per connection. -This variable controls the high-priority queue policy. Assigns the selected number of tickets to each new connection to enter the high-priority queue. Setting this variable to `0` disables the high-priority queue. +* Allowed values: 0 to 4294967295 -### `thread_pool_max_threads` +* Description: Each time a connection uses high-priority scheduling, one ticket is + consumed. -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global | -| Dynamic: | Yes | -| Data type: | Numeric | -| Default value: | 100000 | +* Default: 4294967295 -This variable can limit the maximum number of threads in the pool. When the limit is reached, the server does not create new threads. +* This setting can be configured statically. -### `thread_pool_oversubscribe` +* This setting can be configured dynamically. -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global | -| Dynamic: | Yes | -| Data type: | Numeric | -| Default value: | 3 | +* This change applies only to new client connections. -Determines the number of threads run simultaneously. A value lower than `3` could cause sleep and wake-up actions. -### `thread_pool_size` -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global | -| Dynamic: | Yes | -| Data type: | Numeric | -| Default value: | Number of processors | +### `thread_pool_high_prio_mode` + +Controls which queries qualify for high-priority scheduling. + +* Allowed values: `transactions`, `statements`, `none` + +* Description: Use `transactions` to prioritize active transactions, or `statements` + to prioritize all statements from ticketed connections. + +* Default: `transactions` + +* This setting can be configured statically. + +* This setting can be configured dynamically. + +* This change affects new sessions only. Existing sessions retain the previous mode + until reconnect. -Defines the number of threads that can use the CPU simultaneously. ### `thread_pool_stall_limit` -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global | -| Dynamic: | No | -| Data type: | Numeric | -| Default value: | 500 (ms) | +Controls how long to wait before rescheduling stalled queries. -Defines the number of milliseconds before a running thread is considered stalled. When this limit is reached, the thread pool will wake up or create another thread. This variable prevents a long-running query from monopolizing the pool. +* Allowed values: Integer ≥ 1 (milliseconds) -### `extra_port` +* Description: If a query remains blocked beyond this time, the pool may redistribute + it to another group. -The variable was removed in [Percona Server for MySQL 8.0.14](release-notes/Percona-Server-8.0.14.md). +* Default: 6 -It specifies an additional port that Percona Server for MySQL listens to. This port can be used in case no new connections can be established -due to all worker threads being busy or being locked when the `pool-of-threads` feature is enabled. +* This setting can be configured statically. -The following command connects to the extra port: +* This setting can be configured dynamically. -```shell -mysql --port='extra-port-number' --protocol=tcp -``` +* This setting helps improve query responsiveness under unbalanced workloads. + + +### `thread_pool_min_threads` + +Defines the minimum number of worker threads per group. + +* Allowed values: Integer ≥ 1 -### `extra_max_connections` +* Description: Sets a floor for thread availability within each group. + +* Default: 4 + +* This value is not dynamic. + + + +### `thread_pool_max_threads` -| Option | Description | -| -------------- | ------------------ | -| Command-line: | Yes | -| Config file: | Yes | -| Scope: | Global | -| Dynamic: | No | -| Data type: | Numeric | -| Default value: | 1 | +Defines the maximum number of worker threads per group. -The variable was removed in [Percona Server for MySQL 8.0.14](release-notes/Percona-Server-8.0.14.md). This variable can be used to specify the maximum allowed number of connections -plus one extra `SUPER` user connection on the extra_port. This -can be used with the extra_port variable to access the server in -case no new connections can be established due to all worker threads being busy -or being locked when the `pool-of-threads` feature is enabled. +* Allowed values: Integer ≥ `thread_pool_min_threads` -## Status variables +* Description: Controls upper bound on thread creation. If reached, new queries wait + in the queue. -### `Threadpool_idle_threads` +* Default: 1000 -| Option | Description | -| -------------- | ------------------ | -| Scope: | Global | -| Data type: | Numeric | +* This setting is not dynamic and requires a server restart. -This status variable shows the number of idle threads in the pool. -### `Threadpool_threads` -| Option | Description | -| -------------- | ------------------ | -| Scope: | Global | -| Data type: | Numeric | +Use these variables together to tune concurrency, queue behavior, and scheduling +control based on your workload and connection volume. -This status variable shows the number of threads in the pool.