Flink CDC 2.4.0 无法用多个并行任务进行全量导入 #2275
Replies: 1 comment
-
已经发现问题原因了,因为把mode 调成了earliest,所以没有经过snapshot过程,自始至终都是单一在跑,谢谢。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello:
运行了官方文档的代码
`
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import com.ververica.cdc.debezium.JsonDebeziumDeserializationSchema;
import com.ververica.cdc.connectors.mysql.source.MySqlSource;
public class MySqlSourceExample {
public static void main(String[] args) throws Exception {
MySqlSource mySqlSource = MySqlSource.builder()
.hostname("yourHostname")
.port(yourPort)
.databaseList("yourDatabaseName")
.tableList("yourDatabaseName.yourTableName")
.username("yourUsername")
.password("yourPassword")
.deserializer(new JsonDebeziumDeserializationSchema())
.build();
}
}
`
但是在log中发现,只有一条
Connected to <ip>:3306 at /4 (sid:6348, cid:196)
然后我把set parallelism 换成不同数字,1,2,4等。发现,吞吐量是一样的。期待中的,高并发飞速snapshot并没有发生。我怀疑只有一个任务在做snapshot,而不是多个。
请问是哪里调整的不对,没有开启parallel sync snapshot?
谢谢。
Beta Was this translation helpful? Give feedback.
All reactions