Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tiproxy: update traffic replay docs #19548

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -772,6 +772,7 @@
- [`BEGIN`](/sql-statements/sql-statement-begin.md)
- [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md)
- [`CANCEL IMPORT JOB`](/sql-statements/sql-statement-cancel-import-job.md)
- [`CANCEL TRAFFIC JOBS`](/sql-statements/sql-statement-cancel-traffic-jobs.md)
- [`COMMIT`](/sql-statements/sql-statement-commit.md)
- [`CREATE BINDING`](/sql-statements/sql-statement-create-binding.md)
- [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md)
Expand Down Expand Up @@ -878,13 +879,16 @@
- [`SHOW TABLE REGIONS`](/sql-statements/sql-statement-show-table-regions.md)
- [`SHOW TABLE STATUS`](/sql-statements/sql-statement-show-table-status.md)
- [`SHOW TABLES`](/sql-statements/sql-statement-show-tables.md)
- [`SHOW TRAFFIC JOBS`](/sql-statements/sql-statement-show-traffic-jobs.md)
- [`SHOW VARIABLES`](/sql-statements/sql-statement-show-variables.md)
- [`SHOW WARNINGS`](/sql-statements/sql-statement-show-warnings.md)
- [`SHUTDOWN`](/sql-statements/sql-statement-shutdown.md)
- [`SPLIT REGION`](/sql-statements/sql-statement-split-region.md)
- [`START TRANSACTION`](/sql-statements/sql-statement-start-transaction.md)
- [`TABLE`](/sql-statements/sql-statement-table.md)
- [`TRACE`](/sql-statements/sql-statement-trace.md)
- [`TRAFFIC CAPTURE`](/sql-statements/sql-statement-traffic-capture.md)
- [`TRAFFIC REPLAY`](/sql-statements/sql-statement-traffic-replay.md)
- [`TRUNCATE`](/sql-statements/sql-statement-truncate.md)
- [`UNLOCK STATS`](/sql-statements/sql-statement-unlock-stats.md)
- [`UPDATE`](/sql-statements/sql-statement-update.md)
Expand Down
31 changes: 26 additions & 5 deletions enable-tls-between-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,23 @@ aliases: ['/docs-cn/dev/enable-tls-between-components/','/docs-cn/dev/how-to/sec
cdc server --pd=https://127.0.0.1:2379 --log-file=ticdc.log --addr=0.0.0.0:8301 --advertise-addr=127.0.0.1:8301 --ca=/path/to/ca.pem --cert=/path/to/ticdc-cert.pem --key=/path/to/ticdc-key.pem
```

- TiProxy

在 `config` 文件中设置,并设置相应的 URL 为 https:

```toml
[security]
[server-http-tls]
ca = "/path/to/ca.pem"
cert = "/path/to/tiproxy-server.pem"
key = "/path/to/tiproxy-server-key.pem"
```

此时 TiDB 集群各个组件间已开启加密传输。

> **注意:**
>
> 若 TiDB 集群各个组件间开启加密传输后,在使用 tidb-ctl、tikv-ctlpd-ctl 工具连接集群时,需要指定 client 证书,示例:
> 若 TiDB 集群各个组件间开启加密传输后,在使用 tidb-ctl、tikv-ctlpd-ctl 或 tiproxyctl 工具连接集群时,需要指定 client 证书,示例:

{{< copyable "shell-regular" >}}

Expand Down Expand Up @@ -150,7 +162,7 @@ aliases: ['/docs-cn/dev/enable-tls-between-components/','/docs-cn/dev/how-to/sec

```toml
[security]
cluster-verify-cn = ["tidb", "test-client", "prometheus"]
cluster-verify-cn = ["tidb", "tiproxy", "test-client", "prometheus"]
```

- TiKV
Expand All @@ -168,7 +180,7 @@ aliases: ['/docs-cn/dev/enable-tls-between-components/','/docs-cn/dev/how-to/sec

```toml
[security]
cert-allowed-cn = ["tidb", "pd", "tikv", "tiflash", "test-client", "prometheus"]
cert-allowed-cn = ["tidb", "pd", "tikv", "tiflash", "tiproxy", "test-client", "prometheus"]
```

- TiFlash(从 v4.0.5 版本开始引入)
Expand All @@ -187,10 +199,19 @@ aliases: ['/docs-cn/dev/enable-tls-between-components/','/docs-cn/dev/how-to/sec
cert-allowed-cn = ["tidb", "tikv", "tiflash", "prometheus"]
```

- TiProxy(从 v1.4.0 版本开始引入)

在 `config` 文件中设置:

```toml
[security]
cert-allowed-cn = ["tiproxy", "tidb", "test-client", "prometheus"]
```

## 证书重新加载

- 如果 TiDB 集群部署在本地的数据中心,TiDB、PD、TiKV、TiFlash、TiCDC 和各种 client 在每次新建相互通讯的连接时都会重新读取当前的证书和密钥文件内容,实现证书和密钥的重新加载,无需重启 TiDB 集群。
- 如果 TiDB 集群部署在自己管理的 Cloud,TLS 证书的签发需要与云服务商的证书管理服务集成,TiDB、PD、TiKV、TiFlash、TiCDC 组件的 TLS 证书支持自动轮换,无需重启 TiDB 集群。
- 如果 TiDB 集群部署在本地的数据中心,TiDB、PD、TiKV、TiFlash、TiCDC、TiProxy 和各种 client 在每次新建相互通讯的连接时都会重新读取当前的证书和密钥文件内容,实现证书和密钥的重新加载,无需重启 TiDB 集群。
- 如果 TiDB 集群部署在自己管理的 Cloud,TLS 证书的签发需要与云服务商的证书管理服务集成,TiDB、PD、TiKV、TiFlash、TiCDC、TiProxy 组件的 TLS 证书支持自动轮换,无需重启 TiDB 集群。

## 证书有效期

Expand Down
Binary file modified media/tiproxy/tiproxy-balance-label.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions privilege-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,8 @@ SHOW GRANTS FOR `rw_user`@`192.168.%`;
* `RESTRICTED_USER_ADMIN` 不允许在 SEM 打开的情况下使用 `SUPER` 用户撤销访问权限。
* `RESTRICTED_CONNECTION_ADMIN` 允许 KILL 属于 `RESTRICTED_USER_ADMIN` 用户的连接。该权限对 `KILL` 和 `KILL TIDB` 语句生效。
* `RESTRICTED_REPLICA_WRITER_ADMIN` 允许权限拥有者在 TiDB 集群开启了只读模式的情况下不受影响地执行写入或更新操作,详见 [`tidb_restricted_read_only` 配置项](/system-variables.md#tidb_restricted_read_only-从-v520-版本开始引入)。
* `TRAFFIC_CAPTURE_ADMIN` 允许执行、查看和取消流量捕获任务。详见 [TiProxy 流量回放](/tiproxy/tiproxy-traffic-replay.md)。
* `TRAFFIC_REPLAY_ADMIN` 允许执行、查看和取消流量回放任务。详见 [TiProxy 流量回放](/tiproxy/tiproxy-traffic-replay.md)。

若要查看全部的动态权限,请执行 `SHOW PRIVILEGES` 语句。由于用户可使用插件来添加新的权限,因此可分配的权限列表可能因用户的 TiDB 安装情况而异。

Expand Down Expand Up @@ -506,6 +508,22 @@ SELECT * FROM INFORMATION_SCHEMA.USER_PRIVILEGES WHERE grantee = "'root'@'%'";

当系统变量 [`tidb_resource_control_strict_mode`](/system-variables.md#tidb_resource_control_strict_mode-从-v820-版本开始引入) 设置为 `ON` 时,你需要有 `SUPER` 或者 `RESOURCE_GROUP_ADMIN` 或者 `RESOURCE_GROUP_USER` 权限才能执行该语句。

### TRAFFIC CAPTURE

需要拥有 `SUPER` 或者 `TRAFFIC_CAPTURE_ADMIN` 权限。

### TRAFFIC REPLAY

需要拥有 `SUPER` 或者 `TRAFFIC_REPLAY_ADMIN` 权限。

### CANCEL TAFFIC JOBS

取消捕获任务,需要拥有 `SUPER` 或者 `TRAFFIC_CAPTURE_ADMIN` 权限。取消回放任务,需要拥有 `SUPER` 或者 `TRAFFIC_REPLAY_ADMIN` 权限。

### SHOW TRAFFIC JOBS

查看捕获任务,需要拥有 `SUPER` 或者 `TRAFFIC_CAPTURE_ADMIN` 权限。查看回放任务,需要拥有 `SUPER` 或者 `TRAFFIC_REPLAY_ADMIN` 权限。

## 权限系统的实现

### 授权表
Expand Down
73 changes: 73 additions & 0 deletions sql-statements/sql-statement-cancel-traffic-jobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: CANCEL TRAFFIC JOBS
summary: TiDB 数据库中 CANCEL TRAFFIC JOBS 的使用概况。
---

# CANCEL TRAFFIC JOBS

TiDB v9.0.0 引入了 `CANCEL TRAFFIC JOBS` 语法,用于取消集群中所有 TiProxy 正在执行的流量捕获或回放任务。该操作需要如下权限:

- 取消流量捕获任务,需要有 `SUPER` 或 [`TRAFFIC_CAPTURE_ADMIN`](/privilege-management.md#动态权限) 权限。
- 取消流量回放任务,需要有 `SUPER` 或 [`TRAFFIC_REPLAY_ADMIN`](/privilege-management.md#动态权限) 权限。

## 语法图

```ebnf+diagram
TrafficStmt ::=
"CANCEL" "TRAFFIC" "JOBS"
```

## 示例

例如当前有 2 台 TiProxy 正在流量捕获:

```sql
SHOW TRAFFIC JOBS
```

```
+----------------------------+----------+----------------+---------+----------+---------+-------------+
| START_TIME | END_TIME | INSTANCE | TYPE | PROGRESS | STATUS | FAIL_REASON |
+----------------------------+----------+----------------+---------+----------+---------+-------------+
| 2024-12-17 10:54:41.000000 | | 10.1.0.10:3080 | capture | 45% | running | |
| 2024-12-17 10:54:41.000000 | | 10.1.0.11:3080 | capture | 45% | running | |
+----------------------------+----------+----------------+---------+----------+---------+-------------+
2 rows in set (0.01 sec)
```

取消当前的任务:

```sql
CANCEL TRAFFIC JOBS
```

```
Query OK, 0 rows affected (0.13 sec)
```

再次查看任务,显示任务已被取消:

```sql
SHOW TRAFFIC JOBS
```

```
+----------------------------+----------------------------+----------------+--------+----------+----------+------------------+
| START_TIME | END_TIME | INSTANCE | TYPE | PROGRESS | STATUS | FAIL_REASON |
+----------------------------+----------------------------+----------------+--------+----------+----------+------------------+
| 2024-12-17 10:54:41.000000 | 2024-12-17 11:20:42.000000 | 10.1.0.10:3080 | replay | 45% | canceled | manually stopped |
| 2024-12-17 10:54:41.000000 | 2024-12-17 11:20:42.000000 | 10.1.0.11:3080 | replay | 45% | canceled | manually stopped |
+----------------------------+----------------------------+----------------+--------+----------+----------+------------------+
2 rows in set (0.01 sec)
```

## MySQL 兼容性

该语句是 TiDB 对 MySQL 语法的扩展。

## 另请参阅

* [TiProxy 流量回放](/tiproxy/tiproxy-traffic-replay.md)
* [TRAFFIC CAPTURE](/sql-statements/sql-statement-traffic-capture.md)
* [TRAFFIC REPLAY](/sql-statements/sql-statement-traffic-replay.md)
* [SHOW TRAFFIC JOBS](/sql-statements/sql-statement-show-traffic-jobs.md)
76 changes: 76 additions & 0 deletions sql-statements/sql-statement-show-traffic-jobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: SHOW TRAFFIC JOBS
summary: TiDB 数据库中 SHOW TRAFFIC JOBS 的使用概况。
---

# SHOW TRAFFIC JOBS

TiDB v9.0.0 引入了 `SHOW TRAFFIC JOBS` 语法,用于查看集群中所有 TiProxy 的流量捕获或回放任务。每行代表一台 TiProxy 实例的一个任务。每台 TiProxy 实例最多保存最近的 10 条任务。

当前用户拥有的权限不同,执行该语句显示结果也不同。

- 如果用户有 [`TRAFFIC_CAPTURE_ADMIN`](/privilege-management.md#动态权限) 权限,执行该语句显示流量捕获任务。
- 如果用户有 [`TRAFFIC_REPLAY_ADMIN`](/privilege-management.md#动态权限) 权限,执行该语句显示流量回放任务。
- 如果用户有 `SUPER` 权限或同时具有上述两种权限,执行该语句同时显示流量捕获和流量回放任务。

`SHOW TRAFFIC JOBS` 返回以下列:

| 列名 | 说明 |
| :-------- | :------------- |
| START_TIME | 该任务的开始时间 |
| END_TIME | 如果该任务已结束,该列为结束时间,否则为空 |
| INSTANCE | TiProxy 的实例地址 |
| TYPE | 表示任务类型,`capture` 代表流量捕获任务,`replay` 代表流量回放任务 |
| PROGRESS | 该任务的完成百分比 |
| STATUS | 该任务当前的状态,`running` 代表正在运行,`done` 代表正常完成,`canceled` 代表任务失败 |
| FAIL_REASON | 如果该任务失败,该列为失败的原因,否则为空 |
djshow832 marked this conversation as resolved.
Show resolved Hide resolved

## 语法图

```ebnf+diagram
TrafficStmt ::=
"SHOW" "TRAFFIC" "JOBS"
```

## 示例

查看流量捕获或回放任务:

```sql
SHOW TRAFFIC JOBS
```

下面输出示例表示有 2 台 TiProxy 正在捕获流量,进度都为 45%:

```
+----------------------------+----------+----------------+---------+----------+---------+-------------+
| START_TIME | END_TIME | INSTANCE | TYPE | PROGRESS | STATUS | FAIL_REASON |
+----------------------------+----------+----------------+---------+----------+---------+-------------+
| 2024-12-17 10:54:41.000000 | | 10.1.0.10:3080 | capture | 45% | running | |
| 2024-12-17 10:54:41.000000 | | 10.1.0.11:3080 | capture | 45% | running | |
+----------------------------+----------+----------------+---------+----------+---------+-------------+
2 rows in set (0.01 sec)
```

下面输出示例表示 2 台 TiProxy 的流量回放任务被手动取消:

```
+----------------------------+----------------------------+----------------+--------+----------+----------+------------------+
| START_TIME | END_TIME | INSTANCE | TYPE | PROGRESS | STATUS | FAIL_REASON |
+----------------------------+----------------------------+----------------+--------+----------+----------+------------------+
| 2024-12-17 10:54:41.000000 | 2024-12-17 11:34:42.000000 | 10.1.0.10:3080 | replay | 70% | canceled | manually stopped |
| 2024-12-17 10:54:41.000000 | 2024-12-17 11:34:43.000000 | 10.1.0.11:3080 | replay | 69% | canceled | manually stopped |
+----------------------------+----------------------------+----------------+--------+----------+----------+------------------+
2 rows in set (0.01 sec)
```

## MySQL 兼容性

该语句是 TiDB 对 MySQL 语法的扩展。

## 另请参阅

* [TiProxy 流量回放](/tiproxy/tiproxy-traffic-replay.md)
* [TRAFFIC CAPTURE](/sql-statements/sql-statement-traffic-capture.md)
* [TRAFFIC REPLAY](/sql-statements/sql-statement-traffic-replay.md)
* [CANCEL TRAFFIC JOBS](/sql-statements/sql-statement-cancel-traffic-jobs.md)
67 changes: 67 additions & 0 deletions sql-statements/sql-statement-traffic-capture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: TRAFFIC CAPTURE
summary: TiDB 数据库中 TRAFFIC CAPTURE 的使用概况。
---

# TRAFFIC CAPTURE

TiDB v9.0.0 引入了 `TRAFFIC CAPTURE` 语法,用于向集群中所有 TiProxy 实例发送请求,让 TiProxy 捕获客户端流量到流量文件。

TiProxy 支持捕获流量到本地和外部存储。捕获流量到本地时,需要在捕获流量之后把流量文件手动复制到回放的 TiProxy 集群上,而使用外部存储时则无需手动复制。

TiProxy 支持的外部存储包括 Amazon S3、Google Cloud Storage (GCS)、Azure Blob Storage,或者实现 S3 协议的其他文件存储服务。关于外部存储,请参见[外部存储服务的 URI 格式](/external-storage-uri.md)。

`TRAFFIC CAPTURE` 有以下选项:

- `DURATION`:(必填)指定捕获的时长。可选单位为 `m`(分钟)、`h`(小时)或 `d`(天)。例如 `DURATION="1h"`,表示指定捕获一小时的流量。
- `COMPRESS`:(可选)指定是否压缩流量文件。`true` 表示压缩,压缩格式为 gzip。`false` 表示不压缩。默认值为 `true`。
- `ENCRYPTION_METHOD`:(可选)指定加密流量文件的算法。仅支持 `""`, `plaintext` 和 `aes256-ctr`。`""` 和 `plaintext` 表示不加密,`aes256-ctr` 表示使用 AES256-CTR 算法加密。指定加密时,需要同时配置 [encrytion-key-path](/tiproxy/tiproxy-configuration.md#encryption-key-path)。默认值为 `""`。

捕获流量要求当前用户具有 `SUPER` 或 [`TRAFFIC_CAPTURE_ADMIN`](/privilege-management.md#动态权限) 权限。

## 语法图

```ebnf+diagram
TrafficStmt ::=
"TRAFFIC" "CAPTURE" "TO" stringLit TrafficCaptureOptList

TrafficCaptureOptList ::=
TrafficCaptureOpt
| TrafficCaptureOptList TrafficCaptureOpt

TrafficCaptureOpt ::=
"DURATION" EqOpt stringLit
| "ENCRYPTION_METHOD" EqOpt stringLit
| "COMPRESS" EqOpt Boolean
```

## 示例

捕获 1 天流量到 TiProxy 实例的本地 `/tmp/traffic` 目录:

```sql
TRAFFIC CAPTURE TO "/tmp/traffic" DURATION="1d"
```

捕获 10 分钟流量到 S3:

```sql
TRAFFIC CAPTURE TO "s3://external/traffic?access-key=${access-key}&secret-access-key=${secret-access-key}" DURATION="10m"
```

捕获时,流量文件自动加密,但不自动压缩:

```sql
TRAFFIC CAPTURE TO "/tmp/traffic" DURATION="1h" COMPRESS=false ENCRYPTION_METHOD="aes256-ctr"
```

## MySQL 兼容性

该语句是 TiDB 对 MySQL 语法的扩展。

## 另请参阅

* [TiProxy 流量回放](/tiproxy/tiproxy-traffic-replay.md)
* [TRAFFIC REPLAY](/sql-statements/sql-statement-traffic-replay.md)
* [CANCEL TRAFFIC JOBS](/sql-statements/sql-statement-cancel-traffic-jobs.md)
* [SHOW TRAFFIC JOBS](/sql-statements/sql-statement-show-traffic-jobs.md)
Loading
Loading