diff --git a/ticdc/ticdc-architecture.md b/ticdc/ticdc-architecture.md index bcc7086f0766..c7e98ef30afa 100644 --- a/ticdc/ticdc-architecture.md +++ b/ticdc/ticdc-architecture.md @@ -43,7 +43,7 @@ TiCDC 中的 Changefeed 和 Task 是两个逻辑概念,前者是分配同步 例如: ``` -cdc cli changefeed create --server="http://127.0.0.1:8300" --sink-uri="kafka://127.0.0.1:9092/cdc-test?kafka-version=2.4.0&partition-num=6&max-message-bytes=67108864&replication-factor=1" +cdc cli changefeed create --server="http://127.0.0.1:8300" --sink-uri="kafka://127.0.0.1:9092/cdc-test?kafka-version=${kafka_version}&partition-num=6&max-message-bytes=67108864&replication-factor=1" cat changefeed.toml ...... [sink] diff --git a/ticdc/ticdc-canal-json.md b/ticdc/ticdc-canal-json.md index 681a8f7ec782..53d216ba0d3c 100644 --- a/ticdc/ticdc-canal-json.md +++ b/ticdc/ticdc-canal-json.md @@ -20,7 +20,7 @@ Event 分为三类: 使用 `Canal-JSON` 时的配置样例如下所示: ```shell -cdc cli changefeed create --server=http://127.0.0.1:8300 --changefeed-id="kafka-canal-json" --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0&protocol=canal-json" +cdc cli changefeed create --server=http://127.0.0.1:8300 --changefeed-id="kafka-canal-json" --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}&protocol=canal-json" ``` ## TiDB 扩展字段 @@ -33,7 +33,7 @@ Canal-JSON 协议本是为 MySQL 设计的,其中并不包含 TiDB 专有的 C 配置样例如下所示: ```shell -cdc cli changefeed create --server=http://127.0.0.1:8300 --changefeed-id="kafka-canal-json-enable-tidb-extension" --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0&protocol=canal-json&enable-tidb-extension=true" +cdc cli changefeed create --server=http://127.0.0.1:8300 --changefeed-id="kafka-canal-json-enable-tidb-extension" --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}&protocol=canal-json&enable-tidb-extension=true" ``` ## Message 格式定义 diff --git a/ticdc/ticdc-debezium.md b/ticdc/ticdc-debezium.md index 6344bd592809..b1f428972bcb 100644 --- a/ticdc/ticdc-debezium.md +++ b/ticdc/ticdc-debezium.md @@ -16,7 +16,7 @@ summary: 了解 TiCDC Debezium Protocol 的概念和使用方法。 使用 Debezium 消息格式时的配置样例如下所示: ```shell -cdc cli changefeed create --server=http://127.0.0.1:8300 --changefeed-id="kafka-debezium" --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0&protocol=debezium" +cdc cli changefeed create --server=http://127.0.0.1:8300 --changefeed-id="kafka-debezium" --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}&protocol=debezium" ``` Debezium 输出格式中包含当前行的 Schema 信息,以便下游消费者更好地理解当前行的数据结构。对于不需要输出 Schema 信息的场景,也可以通过在 changefeed 的配置文件或者 `sink-uri` 中将 `debezium-disable-schema` 参数设置为 `true` 来关闭 Schema 信息的输出。 diff --git a/ticdc/ticdc-faq.md b/ticdc/ticdc-faq.md index 1207e14c1f41..9ed1b4e94882 100644 --- a/ticdc/ticdc-faq.md +++ b/ticdc/ticdc-faq.md @@ -111,7 +111,7 @@ TiCDC 为 service GC safepoint 设置的默认存活有效期为 24 小时,即 支持。注意:对于 Canal 协议,TiCDC 只支持 JSON 输出格式,对 protobuf 格式尚未提供官方支持。要开启 Canal 协议的输出,只需在 `--sink-uri` 配置中指定 `protocol` 为 `canal-json` 即可。例如: ```shell -cdc cli changefeed create --server=http://127.0.0.1:8300 --sink-uri="kafka://127.0.0.1:9092/cdc-test?kafka-version=2.4.0&protocol=canal-json" --config changefeed.toml +cdc cli changefeed create --server=http://127.0.0.1:8300 --sink-uri="kafka://127.0.0.1:9092/cdc-test?kafka-version=${kafka_version}&protocol=canal-json" --config changefeed.toml ``` > **注意:** diff --git a/ticdc/ticdc-simple-protocol.md b/ticdc/ticdc-simple-protocol.md index 3b2c4418e21b..f7bde8ff0bd4 100644 --- a/ticdc/ticdc-simple-protocol.md +++ b/ticdc/ticdc-simple-protocol.md @@ -16,7 +16,7 @@ TiCDC Simple Protocol 是一种行级别的数据变更通知协议,为监控 `sink-uri` 配置: ```shell ---sink-uri = "kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0" +--sink-uri = "kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}" ``` changefeed 配置: diff --git a/ticdc/ticdc-sink-to-kafka.md b/ticdc/ticdc-sink-to-kafka.md index a9af2e0ea238..2ad02b340c8c 100644 --- a/ticdc/ticdc-sink-to-kafka.md +++ b/ticdc/ticdc-sink-to-kafka.md @@ -14,14 +14,14 @@ summary: 了解如何使用 TiCDC 将数据同步到 Kafka。 ```shell cdc cli changefeed create \ --server=http://10.0.10.25:8300 \ - --sink-uri="kafka://127.0.0.1:9092/topic-name?protocol=canal-json&kafka-version=2.4.0&partition-num=6&max-message-bytes=67108864&replication-factor=1" \ + --sink-uri="kafka://127.0.0.1:9092/topic-name?protocol=canal-json&kafka-version=${kafka_version}&partition-num=6&max-message-bytes=67108864&replication-factor=1" \ --changefeed-id="simple-replication-task" ``` ```shell Create changefeed successfully! ID: simple-replication-task -Info: {"sink-uri":"kafka://127.0.0.1:9092/topic-name?protocol=canal-json&kafka-version=2.4.0&partition-num=6&max-message-bytes=67108864&replication-factor=1","opts":{},"create-time":"2023-11-28T22:04:08.103600025+08:00","start-ts":415241823337054209,"target-ts":0,"admin-job-type":0,"sort-engine":"unified","sort-dir":".","config":{"case-sensitive":false,"filter":{"rules":["*.*"],"ignore-txn-start-ts":null,"ddl-allow-list":null},"mounter":{"worker-num":16},"sink":{"dispatchers":null},"scheduler":{"type":"table-number","polling-time":-1}},"state":"normal","history":null,"error":null} +Info: {"sink-uri":"kafka://127.0.0.1:9092/topic-name?protocol=canal-json&kafka-version=${kafka_version}&partition-num=6&max-message-bytes=67108864&replication-factor=1","opts":{},"create-time":"2023-11-28T22:04:08.103600025+08:00","start-ts":415241823337054209,"target-ts":0,"admin-job-type":0,"sort-engine":"unified","sort-dir":".","config":{"case-sensitive":false,"filter":{"rules":["*.*"],"ignore-txn-start-ts":null,"ddl-allow-list":null},"mounter":{"worker-num":16},"sink":{"dispatchers":null},"scheduler":{"type":"table-number","polling-time":-1}},"state":"normal","history":null,"error":null} ``` - `--server`:TiCDC 集群中任意一个 TiCDC 服务器的地址。 @@ -63,7 +63,7 @@ Sink URI 用于指定 TiCDC 目标系统的连接信息,遵循以下格式: 一个通用的配置样例如下所示: ```shell ---sink-uri="kafka://127.0.0.1:9092/topic-name?protocol=canal-json&kafka-version=2.4.0&partition-num=6&max-message-bytes=67108864&replication-factor=1" +--sink-uri="kafka://127.0.0.1:9092/topic-name?protocol=canal-json&kafka-version=${kafka_version}&partition-num=6&max-message-bytes=67108864&replication-factor=1" ``` URI 中可配置的的参数如下: @@ -123,7 +123,7 @@ URI 中可配置的的参数如下: - SASL/PLAIN ```shell - --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0&sasl-user=alice-user&sasl-password=alice-secret&sasl-mechanism=plain" + --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}&sasl-user=alice-user&sasl-password=alice-secret&sasl-mechanism=plain" ``` - SASL/SCRAM @@ -135,7 +135,7 @@ URI 中可配置的的参数如下: SASL/GSSAPI `user` 类型认证: ```shell - --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0&sasl-mechanism=gssapi&sasl-gssapi-auth-type=user&sasl-gssapi-kerberos-config-path=/etc/krb5.conf&sasl-gssapi-service-name=kafka&sasl-gssapi-user=alice/for-kafka&sasl-gssapi-password=alice-secret&sasl-gssapi-realm=example.com" + --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}&sasl-mechanism=gssapi&sasl-gssapi-auth-type=user&sasl-gssapi-kerberos-config-path=/etc/krb5.conf&sasl-gssapi-service-name=kafka&sasl-gssapi-user=alice/for-kafka&sasl-gssapi-password=alice-secret&sasl-gssapi-realm=example.com" ``` `sasl-gssapi-user` 和 `sasl-gssapi-realm` 的值与 kerberos 中指定的 [principle](https://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-user/What-is-a-Kerberos-Principal_003f.html) 有关。例如,principle 为 `alice/for-kafka@example.com`,则 `sasl-gssapi-user` 和 `sasl-gssapi-realm` 的值应该分别指定为 `alice/for-kafka` 和 `example.com`。 @@ -143,7 +143,7 @@ URI 中可配置的的参数如下: SASL/GSSAPI `keytab` 类型认证: ```shell - --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=2.4.0&sasl-mechanism=gssapi&sasl-gssapi-auth-type=keytab&sasl-gssapi-kerberos-config-path=/etc/krb5.conf&sasl-gssapi-service-name=kafka&sasl-gssapi-user=alice/for-kafka&sasl-gssapi-keytab-path=/var/lib/secret/alice.key&sasl-gssapi-realm=example.com" + --sink-uri="kafka://127.0.0.1:9092/topic-name?kafka-version=${kafka_version}&sasl-mechanism=gssapi&sasl-gssapi-auth-type=keytab&sasl-gssapi-kerberos-config-path=/etc/krb5.conf&sasl-gssapi-service-name=kafka&sasl-gssapi-user=alice/for-kafka&sasl-gssapi-keytab-path=/var/lib/secret/alice.key&sasl-gssapi-realm=example.com" ``` SASL/GSSAPI 认证方式详见 [Configuring GSSAPI](https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_gssapi.html)。 diff --git a/ticdc/troubleshoot-ticdc.md b/ticdc/troubleshoot-ticdc.md index 4dfcb038f48d..6a603bf71f3c 100644 --- a/ticdc/troubleshoot-ticdc.md +++ b/ticdc/troubleshoot-ticdc.md @@ -135,5 +135,5 @@ cdc cli changefeed create --server=http://127.0.0.1:8300 --sink-uri="mysql://roo 该问题通常是由于 TiCDC 与 Kafka 集群连接失败导致。你可以通过检查 Kafka 的日志以及网络状况来排查。一个常见的原因是在创建同步任务时没有指定正确的 `kafka-version` 参数,导致 TiCDC 内部的 Kafka client 在访问 Kafka server 时使用了错误的 Kafka API 版本。你可以通过配置 [`--sink-uri`](/ticdc/ticdc-sink-to-kafka.md#sink-uri-配置-kafka) 指定正确的 `kafka-version` 参数来修复。例如: ```shell -cdc cli changefeed create --server=http://127.0.0.1:8300 --sink-uri "kafka://127.0.0.1:9092/test?topic=test&protocol=open-protocol&kafka-version=2.4.0" +cdc cli changefeed create --server=http://127.0.0.1:8300 --sink-uri "kafka://127.0.0.1:9092/test?topic=test&protocol=open-protocol&kafka-version=${kafka_version}" ```