apache/seatunnel

[Bug] [Postgres-CDC] Failed to synchronize partitioned tables to MySQL using Postgres-CDC

Open

#10.593 aperta il 12 mar 2026

Vedi su GitHub
 (10 commenti) (0 reazioni) (1 assegnatario)Java (1432 fork)batch import
buggood first issuehelp wanted

Metriche repository

Star
 (6897 star)
Metriche merge PR
 (Merge medio 13g 21h) (143 PR mergiate in 30 g)

Descrizione

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

我通过/hazelcast/rest/maps/submit-job进行提交任务,现在测试阶段打算把一个Postgres的分区表(pg_partman自动分区)的数据以流处理的方式同步到MySQL中,hocon配置如下

env {
    job.mode = "STREAMING"
    execution.parallelism = 1
    checkpoint.interval = 10000
}

source {
    Postgres-CDC {
        url = "jdbc:postgresql://172.31.204.230:5432/datahub"
        username = "datahub"
        password = "1111"
        database-names = ["datahub"]
        schema-names = ["original"]
        table-names = ["datahub.ods.lazada_order_details"]

        startup.mode = "latest"
        slot.name = "seatunnel_slot"
        decoding.plugin.name = "pgoutput"
    }
}

transform {

}

sink {
    Jdbc {
        url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
        driver = "com.mysql.cj.jdbc.Driver"
        username = "root"
        password = "111"

        generate_sink_sql = true
        database = "shop_fox_prod"
        table = "lazada_order_details"

        primary_keys = ["id", "created_at"]
    }
}

在web界面能看到任务长期处于RUNNING状态,但是source的TableSourceReceivedCount字段始终为0,sink的TableSinkWriteCount字段也为0,目前使用的是apache/seatunnel:2.3.13的docker镜像,默认使用Zeta引擎,

然后观察日志发现有注意到:SELECT reltuples FROM pg_class r WHERE relkind = 'r' AND relname = 'lazada_order_details'; 的SQL,是因为现在的Postgres-CDC只能同步普通表吗?

SeaTunnel Version

2.3.13

SeaTunnel Config

env {
    job.mode = "STREAMING"
    execution.parallelism = 1
    checkpoint.interval = 10000
}

source {
    Postgres-CDC {
        url = "jdbc:postgresql://172.31.204.230:5432/datahub"
        username = "datahub"
        password = "TeKvkjtFtBceyCF6"
        database-names = ["datahub"]
        schema-names = ["original"]
        table-names = ["datahub.original.lazada_order_details"]

        startup.mode = "latest"
        slot.name = "seatunnel_slot"
        decoding.plugin.name = "pgoutput"
    }
}

transform {

}

sink {
    Jdbc {
        url = "jdbc:mysql://172.31.204.227:3306/shop_fox_prod?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true"
        driver = "com.mysql.cj.jdbc.Driver"
        username = "root"
        password = "6kF75pjtsauaWCuy"

        generate_sink_sql = true
        database = "shop_fox_prod"
        table = "lazada_order_details"

        primary_keys = ["id", "created_at"]
    }
}

Running Command

http://172.31.204.230:25801/hazelcast/rest/maps/submit-job?format=hocon&jobName=LazadaOrderDetails

Error Exception

null

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guida contributor