diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index b3f007c8976a1..f0d504f303d53 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -409,7 +409,8 @@ Expected output: > **Note:** > -> If you specify `--full-backup-storage` as the incremental backup address for `restore point`, for restores of this backup and any previous incremental backups, you need to set the parameter `--allow-pitr-from-incremental` to `true` to make the incremental backups compatible with the subsequent log backups. +> - If you specify `--full-backup-storage` as the incremental backup address for `restore point`, for restores of this backup and any previous incremental backups, you need to set the parameter `--allow-pitr-from-incremental` to `true` to make the incremental backups compatible with the subsequent log backups. +> - For information about checksum configuration, see [Checksum](/br/br-snapshot-manual.md#checksum). You can run the `tiup br restore point` command to perform a PITR on a new cluster or just restore the log backup data. @@ -516,4 +517,4 @@ If any of the above conditions are not met, or if you need to perform a point-in > **Note:** > -> When restoring a log backup that contains records of snapshot (full) restore data, you must use BR v9.0.0 or later. Otherwise, restoring the recorded full restore data might fail. \ No newline at end of file +> When restoring a log backup that contains records of snapshot (full) restore data, you must use BR v9.0.0 or later. Otherwise, restoring the recorded full restore data might fail. diff --git a/br/br-snapshot-manual.md b/br/br-snapshot-manual.md index aeccb16e55a6c..c24dd06dadb0d 100644 --- a/br/br-snapshot-manual.md +++ b/br/br-snapshot-manual.md @@ -21,6 +21,7 @@ This document describes the commands of TiDB snapshot backup and restore accordi - [Restore multiple tables with table filter](#restore-multiple-tables-with-table-filter) - [Restore execution plan bindings from the `mysql` schema](#restore-execution-plan-bindings-from-the-mysql-schema) - [Restore encrypted snapshots](#restore-encrypted-snapshots) +- [Checksum](#checksum) For more information about snapshot backup and restore, refer to: @@ -48,8 +49,7 @@ In the preceding command: > **Note:** > -> - Starting from v8.5.0, the BR tool disables the table-level checksum calculation during full backups by default (`--checksum=false`) to improve backup performance. -> - The BR tool already supports self-adapting to GC. It automatically registers `backupTS` (the latest PD timestamp by default) to PD's `safePoint` to ensure that TiDB's GC Safe Point does not move forward during the backup, thus avoiding manually setting GC configurations. +> The BR tool already supports self-adapting to GC. It automatically registers `backupTS` (the latest PD timestamp by default) to PD's `safePoint` to ensure that TiDB's GC Safe Point does not move forward during the backup, thus avoiding manually setting GC configurations. During backup, a progress bar is displayed in the terminal, as shown below. When the progress bar advances to 100%, the backup is complete. @@ -178,7 +178,7 @@ In the preceding command: - `--ratelimit`: The maximum speed **per TiKV** performing restore tasks. The unit is in MiB/s. - `--log-file`: The target file where the `br` log is written. -During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed. Then `br` will verify the restored data to ensure data security. +During restore, a progress bar is displayed in the terminal as shown in the following example. When the progress bar advances to 100%, the restore task is completed. After the restoration is complete, if table-level [checksum](#checksum) is enabled, the BR tool performs table data verification to ensure the logical integrity of the data. Note that file-level checksums are always performed to ensure the basic integrity of the restored files. ```shell Split&Scatter Region <--------------------------------------------------------------------> 100.00% @@ -287,3 +287,46 @@ tiup br restore full\ --crypter.method aes128-ctr \ --crypter.key 0123456789abcdef0123456789abcdef ``` + +## Checksum + +Checksum is a method used by the BR tool to verify the integrity of backup and restore data. BR supports two levels of checksums: + +1. **File-level checksum**: Verifies the backup files themselves to ensure integrity during storage and transmission. This checksum is always enabled and cannot be disabled. +2. **Table-level checksum**: Verifies the integrity of table data content and confirms the business logic consistency of the data. This checksum is disabled by default but can be enabled through parameters. + +Balancing performance and security considerations, BR handles table-level checksums as follows: + +### Backup checksum + +Starting from v8.5.0, when performing full backups, the BR tool does not perform table-level checksum verification (`--checksum=false`) by default to improve backup performance. If you need to perform table-level checksums during backup, you can explicitly specify `--checksum=true`. File-level checksums are always performed to ensure the integrity of backup files. + +Performing table-level checksums can verify data integrity during backup but increases backup time. In most cases, it is safe to use the default setting (that is, table-level checksum is disabled) to improve backup speed. + +### Restore checksum + +Starting from v9.0.0, the BR tool does not perform table-level checksum verification (`--checksum=false`) by default during restore operations to improve restore performance. If you need to perform table-level checksum verification, you can explicitly specify `--checksum=true`. File-level checksum verification is always performed to ensure the basic integrity of restored data. + +After restoration, data validation is usually performed to ensure data security. When table-level checksums are disabled, the comprehensive validation step for table data is skipped, thereby accelerating the restore process. For scenarios with strict data integrity requirements, you can enable table-level checksums. + +### Checksum configuration examples + +Enable table-level checksums during backup: + +```shell +tiup br backup full \ + --pd "${PD_IP}:2379" \ + --storage "s3://${backup_collection_addr}/snapshot-${date}?access-key=${access-key}&secret-access-key=${secret-access-key}" \ + --checksum=true \ + --log-file backupfull.log +``` + +Enable table-level checksums during restore: + +```shell +tiup br restore full \ + --pd "${PD_IP}:2379" \ + --storage "s3://${backup_collection_addr}/snapshot-${date}?access-key=${access-key}&secret-access-key=${secret-access-key}" \ + --checksum=true \ + --log-file restorefull.log +``` diff --git a/br/use-br-command-line-tool.md b/br/use-br-command-line-tool.md index a92f39e135a2a..a39d587d3feaf 100644 --- a/br/use-br-command-line-tool.md +++ b/br/use-br-command-line-tool.md @@ -62,6 +62,7 @@ A `tiup br` command consists of multiple layers of sub-commands. Currently, br c * `--tikv-max-restore-concurrency`: the maximum number of concurrent tasks per TiKV node during snapshot restore. * `--compression`: determines the compression algorithm used for generating backup files. It supports `lz4`, `snappy`, and `zstd`, with the default being `zstd` (usually no need to modify). For guidance on choosing different compression algorithms, refer to [this document](https://github.com/EighteenZi/rocksdb_wiki/blob/master/Compression.md). * `--compression-level`: sets the compression level corresponding to the chosen compression algorithm for backup. The default compression level for `zstd` is 3. In most cases there is no need to set this option. +* `--checksum`: controls whether to perform table-level checksum verification during backup and restore. The default value is `false`, meaning that checksum verification is not performed. For more details, see [Checksum](/br/br-snapshot-manual.md#checksum). ## Commands of full backup