Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canal 大量统一消息内容重复推送,导致消息延迟 #5196

Open
ssp1523 opened this issue Jul 10, 2024 · 3 comments
Open

Canal 大量统一消息内容重复推送,导致消息延迟 #5196

ssp1523 opened this issue Jul 10, 2024 · 3 comments

Comments

@ssp1523
Copy link

ssp1523 commented Jul 10, 2024

Question

消息内容:

{"data":null,"database":"test","es":1720514631000,"gtid":"","id":213,"isDdl":true,"mysqlType":null,"old":null,"pkNames":null,"sql":"/* rds internal mark */CREATE DATABASE IF NOT EXISTS test","sqlType":null,"table":"","ts":1720605940595,"type":"QUERY"}

这个消息内容一直在重复推送,导致正常的表数据变更延迟高到1天以上,而且延迟时间一直在增长,使用的是1.1.8版本

@agapple
Copy link
Member

agapple commented Jul 16, 2024

是不是有canal消费的其他报错,导致一直在做retry,找一下root cause

@Beauxie
Copy link

Beauxie commented Jul 16, 2024

请问这个问题解决了吗?目前我也遇到了

@JunierJJJ
Copy link

JunierJJJ commented Aug 19, 2024

我也遇到了 有一条消息一直在重复发生 然后server端有报如下错误
ERROR c.a.o.canal.connector.kafka.producer.CanalKafkaProducer - java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.NotEnoughReplicasException: Messages are rejected since there are fewer in-sync replicas than required.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.NotEnoughReplicasException: Messages are rejected since there are fewer in-sync replicas than required.
at com.alibaba.otter.canal.connector.kafka.producer.CanalKafkaProducer.send(CanalKafkaProducer.java:184) ~[na:na]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants