apache/seatunnel
View on GitHub[Bug] [Postgres-CDC] Failed to synchronize partitioned tables to MySQL using Postgres-CDC
Open
#10593 opened on Mar 12, 2026
buggood first issuehelp wanted
Description
Search before asking
- I had searched in the issues and found no similar issues.
What happened
我通过/hazelcast/rest/maps/submit-job进行提交任务,现在测试阶段打算把一个Postgres的分区表(pg_partman自动分区)的数据以流处理的方式同步到MySQL中,hocon配置如下
env {
job.mode = "STREAMING"
execution.parallelism = 1
checkpoint.interval = 10000
}
source {
Postgres-CDC {
url = "jdbc:postgresql://172.31.204.230:5432/datahub"
username = "datahub"
password = "1111"
database-names = ["datahub"]
schema-names = ["original"]
table-names = ["datahub.ods.lazada_order_details"]
startup.mode = "latest"
slot.name = "seatunnel_slot"
decoding.plugin.name = "pgoutput"
}
}
transform {
}
sink {
Jdbc {
url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
driver = "com.mysql.cj.jdbc.Driver"
username = "root"
password = "111"
generate_sink_sql = true
database = "shop_fox_prod"
table = "lazada_order_details"
primary_keys = ["id", "created_at"]
}
}
在web界面能看到任务长期处于RUNNING状态,但是source的TableSourceReceivedCount字段始终为0,sink的TableSinkWriteCount字段也为0,目前使用的是apache/seatunnel:2.3.13的docker镜像,默认使用Zeta引擎,
然后观察日志发现有注意到:SELECT reltuples FROM pg_class r WHERE relkind = 'r' AND relname = 'lazada_order_details'; 的SQL,是因为现在的Postgres-CDC只能同步普通表吗?
SeaTunnel Version
2.3.13
SeaTunnel Config
env {
job.mode = "STREAMING"
execution.parallelism = 1
checkpoint.interval = 10000
}
source {
Postgres-CDC {
url = "jdbc:postgresql://172.31.204.230:5432/datahub"
username = "datahub"
password = "TeKvkjtFtBceyCF6"
database-names = ["datahub"]
schema-names = ["original"]
table-names = ["datahub.original.lazada_order_details"]
startup.mode = "latest"
slot.name = "seatunnel_slot"
decoding.plugin.name = "pgoutput"
}
}
transform {
}
sink {
Jdbc {
url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
driver = "com.mysql.cj.jdbc.Driver"
username = "root"
password = "6kF75pjtsauaWCuy"
generate_sink_sql = true
database = "shop_fox_prod"
table = "lazada_order_details"
primary_keys = ["id", "created_at"]
}
}
Running Command
http://172.31.204.230:25801/hazelcast/rest/maps/submit-job?format=hocon&jobName=LazadaOrderDetails
Error Exception
null
Zeta or Flink or Spark Version
No response
Java or Scala Version
No response
Screenshots
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct