apache/seatunnel

[Bug] [Postgres-CDC] Failed to synchronize partitioned tables to MySQL using Postgres-CDC

Open

#10593 opened on Mar 12, 2026

View on GitHub
 (10 comments) (0 reactions) (1 assignee)Java (6,897 stars) (1,432 forks)batch import
buggood first issuehelp wanted

Description

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

我通过/hazelcast/rest/maps/submit-job进行提交任务,现在测试阶段打算把一个Postgres的分区表(pg_partman自动分区)的数据以流处理的方式同步到MySQL中,hocon配置如下

env {
    job.mode = "STREAMING"
    execution.parallelism = 1
    checkpoint.interval = 10000
}

source {
    Postgres-CDC {
        url = "jdbc:postgresql://172.31.204.230:5432/datahub"
        username = "datahub"
        password = "1111"
        database-names = ["datahub"]
        schema-names = ["original"]
        table-names = ["datahub.ods.lazada_order_details"]

        startup.mode = "latest"
        slot.name = "seatunnel_slot"
        decoding.plugin.name = "pgoutput"
    }
}

transform {

}

sink {
    Jdbc {
        url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
        driver = "com.mysql.cj.jdbc.Driver"
        username = "root"
        password = "111"

        generate_sink_sql = true
        database = "shop_fox_prod"
        table = "lazada_order_details"

        primary_keys = ["id", "created_at"]
    }
}

在web界面能看到任务长期处于RUNNING状态,但是source的TableSourceReceivedCount字段始终为0,sink的TableSinkWriteCount字段也为0,目前使用的是apache/seatunnel:2.3.13的docker镜像,默认使用Zeta引擎,

然后观察日志发现有注意到:SELECT reltuples FROM pg_class r WHERE relkind = 'r' AND relname = 'lazada_order_details'; 的SQL,是因为现在的Postgres-CDC只能同步普通表吗?

SeaTunnel Version

2.3.13

SeaTunnel Config

env {
    job.mode = "STREAMING"
    execution.parallelism = 1
    checkpoint.interval = 10000
}

source {
    Postgres-CDC {
        url = "jdbc:postgresql://172.31.204.230:5432/datahub"
        username = "datahub"
        password = "TeKvkjtFtBceyCF6"
        database-names = ["datahub"]
        schema-names = ["original"]
        table-names = ["datahub.original.lazada_order_details"]

        startup.mode = "latest"
        slot.name = "seatunnel_slot"
        decoding.plugin.name = "pgoutput"
    }
}

transform {

}

sink {
    Jdbc {
        url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
        driver = "com.mysql.cj.jdbc.Driver"
        username = "root"
        password = "6kF75pjtsauaWCuy"

        generate_sink_sql = true
        database = "shop_fox_prod"
        table = "lazada_order_details"

        primary_keys = ["id", "created_at"]
    }
}

Running Command

http://172.31.204.230:25801/hazelcast/rest/maps/submit-job?format=hocon&jobName=LazadaOrderDetails

Error Exception

null

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide