Flink CDC MySQL Sync Delay

[6-archer-9]

• The current version of flink-sql-connector-mysql-cdc is 2.4.1
• The metric being used is currentFetchEventTimeLag from the CDC source code. This metric represents the time difference between the point when data is committed in the database and when it enters the Flink CDC source.
• The network transfer between internal machines is stable, with an average network latency of around 3ms as observed through pinging the hosts. However, no specific network latency tests between hosts have been conducted.

Example 1:

• The delay suddenly spiked to 12 seconds and persisted until the arrival of the next data entry.

• The input data to the source is relatively stable, approximately around 120 entries per 2 minutes (2.0 * 60s = 120s).

• The numRecordsInPerSecond metric for the writer operator is illustrated in the following graph:

Example 2:

• By observing the numRecordsInPerSecond metric of the source operator, we discovered a continuous influx of data, but the latency metric remained constant.
  e
• In contrast, the numRecordsInPerSecond metric for the writer operator provides a explanation.

*The factors influencing currentFetchEventTimeLag are challenging to identify. Although I explored MySQL monitoring in Zabbix, I couldn't locate relevant metrics. How can I resolve this issue?