apache/seatunnel

[Bug] [Postgres-CDC] Failed to synchronize partitioned tables to MySQL using Postgres-CDC

Open

#10,593 opened on Mar 12, 2026

View on GitHub
 (10 comments) (0 reactions) (1 assignee)Java (1,432 forks)batch import
buggood first issuehelp wanted

Repository metrics

Stars
 (6,897 stars)
PR merge metrics
 (Avg merge 13d 21h) (143 merged PRs in 30d)

Description

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

我通过/hazelcast/rest/maps/submit-job进行提交任务,现在测试阶段打算把一个Postgres的分区表(pg_partman自动分区)的数据以流处理的方式同步到MySQL中,hocon配置如下

env {
    job.mode = "STREAMING"
    execution.parallelism = 1
    checkpoint.interval = 10000
}

source {
    Postgres-CDC {
        url = "jdbc:postgresql://172.31.204.230:5432/datahub"
        username = "datahub"
        password = "1111"
        database-names = ["datahub"]
        schema-names = ["original"]
        table-names = ["datahub.ods.lazada_order_details"]

        startup.mode = "latest"
        slot.name = "seatunnel_slot"
        decoding.plugin.name = "pgoutput"
    }
}

transform {

}

sink {
    Jdbc {
        url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
        driver = "com.mysql.cj.jdbc.Driver"
        username = "root"
        password = "111"

        generate_sink_sql = true
        database = "shop_fox_prod"
        table = "lazada_order_details"

        primary_keys = ["id", "created_at"]
    }
}

在web界面能看到任务长期处于RUNNING状态,但是source的TableSourceReceivedCount字段始终为0,sink的TableSinkWriteCount字段也为0,目前使用的是apache/seatunnel:2.3.13的docker镜像,默认使用Zeta引擎,

然后观察日志发现有注意到:SELECT reltuples FROM pg_class r WHERE relkind = 'r' AND relname = 'lazada_order_details'; 的SQL,是因为现在的Postgres-CDC只能同步普通表吗?

SeaTunnel Version

2.3.13

SeaTunnel Config

env {
    job.mode = "STREAMING"
    execution.parallelism = 1
    checkpoint.interval = 10000
}

source {
    Postgres-CDC {
        url = "jdbc:postgresql://172.31.204.230:5432/datahub"
        username = "datahub"
        password = "TeKvkjtFtBceyCF6"
        database-names = ["datahub"]
        schema-names = ["original"]
        table-names = ["datahub.original.lazada_order_details"]

        startup.mode = "latest"
        slot.name = "seatunnel_slot"
        decoding.plugin.name = "pgoutput"
    }
}

transform {

}

sink {
    Jdbc {
        url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
        driver = "com.mysql.cj.jdbc.Driver"
        username = "root"
        password = "6kF75pjtsauaWCuy"

        generate_sink_sql = true
        database = "shop_fox_prod"
        table = "lazada_order_details"

        primary_keys = ["id", "created_at"]
    }
}

Running Command

http://172.31.204.230:25801/hazelcast/rest/maps/submit-job?format=hocon&jobName=LazadaOrderDetails

Error Exception

null

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide