[Bug] [MySQL CDC] MySQL cdc start by time，TIMESTAMP startup mode cannot recover from checkpoints. · apache/seatunnel#10899

(3 comments) (0 reactions) (0 assignees)Java (6,897 stars) (1,432 forks)batch import

bughelp wanted

Description

Search before asking

I had searched in the issues and found no similar issues.

What happened

When binlog corruption occurs: anomalies at the storage layer result in incomplete binlog files themselves. SeaTunnel Failure recovery mechanism: When startup.mode = TIMESTAMP, SeaTunnel cannot automatically resume consumption from the last offset. Instead, it reads binlog data starting from the specified time obtained via the configuration parameter startup.timestamp = 17787455400000. If the binlog files have been purged by MySQL (e.g., binlog retention period is set to 7 days while the task has been running for one month), this mode is unable to support resume transmission from breakpoints.

SeaTunnel Version

2.3.13

SeaTunnel Config

env {
  job.name = "test"
  parallelism = 2
  job.mode = "STREAMING"
  checkpoint.interval = 12000
  #checkpoint.interval = 5000
  checkpoint.timeout = 600000
}

source {
  MySQL-CDC {
    base-url = "jdbc:mysql://xxx.xxx.com:3306/test"
    username = "xxx"
    password = "xxx"
    server-time-zone = "UTC"
    connect.timeout.ms = 60000
    connect.max-retries = 3
    int_type_narrowing = false
    table-names = [
        "test.a",
        "test.b",
        "test.c"
    ]
   

    startup.mode = "timestamp"
    startup.timestamp = 1778745540000

    snapshot.split.size = 100000
    schema-changes.enabled = true
  }

}

sink {
    Doris {
        fenodes = "172.xx.xx.xx:8030"
        username = root
        password = "xxxxxx"
        database = "${database_name}"
        table = "${table_name}"
        sink.enable-2pc = "true"
        sink.enable-delete = "true"
        sink.check-interval = 50000
        sink.max-retries = 5
        sink.label-prefix = "manege_aa"
        doris.config = {
            format="json"
            read_json_by_line="true"
        }
        save_mode_create_template = """
            CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` (
               ${rowtype_primary_key},
               ${rowtype_fields}
               ) ENGINE=OLAP
               UNIQUE KEY (${rowtype_primary_key})
               COMMENT '${comment}'
               DISTRIBUTED BY HASH (${rowtype_primary_key})
               PROPERTIES (
                   "replication_allocation" = "tag.location.default: 3",
                   "in_memory" = "false",
                   "storage_format" = "V2",
                   "disable_auto_compaction" = "false"
               )
            """

    }
}

Running Command

./bin/seatunnel-cluster.sh -d

Error Exception

2026-05-18 00:22:34,807 ERROR [ySqlStreamingChangeEventSource] [blc-xxx.xxx.com:3306] - Error during binlog processing. Last offset stored = {transaction_id=null, ts_sec=1779063692, file=mysql-bin-changelog.235929, pos=4835380, server_id=1745163349, event=1}, binlog reader near position = mysql-bin-changelog.235929/4835477
2026-05-18 00:22:34,808 ERROR [i.d.p.ErrorHandler            ] [blc-xxx.xxx.com:3306] - Producer failure
io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1779063692000, eventType=TABLE_MAP, serverId=1745163349, headerLength=19, dataLength=1608, nextPosition=4837104, flags=0}
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1503) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1559) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_341]
Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1779063692000, eventType=TABLE_MAP, serverId=1745163349, headerLength=19, dataLength=1608, nextPosition=4837104, flags=0}
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:313) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:237) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:273) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051) ~[?:?]
	... 3 more
Caused by: java.io.EOFException: Failed to read remaining 750 of 1314 bytes from position -624251094. Block length: 750. Initial block length: 1604.
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.fill(ByteArrayInputStream.java:115) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:105) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:48) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:313) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:237) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:273) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051) ~[?:?]
	... 3 more


2026-05-18 00:22:34,941 ERROR [.s.c.s.r.f.SplitFetcherManager] [Source Data Fetcher for BlockingWorker-TaskGroupLocation{jobId=1107219654020956161, pipelineId=1, taskGroupId=6}] - Received uncaught exception.
java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165) ~[seatunnel-transforms-v2.jar:2.3.13]
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:81) ~[seatunnel-transforms-v2.jar:2.3.13]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_341]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_341]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_341]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_341]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_341]
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1559) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
Caused by: io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1779063692000, eventType=TABLE_MAP, serverId=1745163349, headerLength=19, dataLength=1608, nextPosition=4837104, flags=0}
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1503) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1559) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1779063692000, eventType=TABLE_MAP, serverId=1745163349, headerLength=19, dataLength=1608, nextPosition=4837104, flags=0}
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:313) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:237) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:273) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
Caused by: java.io.EOFException: Failed to read remaining 750 of 1314 bytes from position -624251094. Block length: 750. Initial block length: 1604.
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.fill(ByteArrayInputStream.java:115) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:105) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:48) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:313) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:237) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:273) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
2026-05-18 00:22:34,941 INFO  [r.IncrementalSourceSplitReader] [Source Data Fetcher for BlockingWorker-TaskGroupLocation{jobId=1107219654020956161, pipelineId=1, taskGroupId=6}] - Close current fetcher org.apache.seatunnel.connectors.cdc.base.source.reader.external.IncrementalSourceStreamFetcher
2026-05-18 00:22:34,986 WARN  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1107219654020956161, pipelineId=1, taskGroupId=6}] - [localhost]:5801 [fanqie-manage-seatunnel] [5.1] Exception in org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask@54eab513
java.lang.RuntimeException: One or more fetchers have encountered exception
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:147) ~[seatunnel-transforms-v2.jar:2.3.13]
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:167) ~[seatunnel-transforms-v2.jar:2.3.13]
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:93) ~[seatunnel-transforms-v2.jar:2.3.13]
	at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceReader.pollNext(IncrementalSourceReader.java:119) ~[?:?]
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:114) ~[seatunnel-transforms-v2.jar:2.3.13]
	at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceReader.pollNext(IncrementalSourceReader.java:119) ~[?:?]
	at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:159) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:127) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:165) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:132) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:683) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1012) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43) ~[seatunnel-starter.jar:2.3.13]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_341]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_341]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_341]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_341]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_341]
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165) ~[seatunnel-transforms-v2.jar:2.3.13]
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:81) ~[seatunnel-transforms-v2.jar:2.3.13]
	... 5 more
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1559) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
Caused by: io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1779063692000, eventType=TABLE_MAP, serverId=1745163349, headerLength=19, dataLength=1608, nextPosition=4837104, flags=0}
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1503) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1559) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1779063692000, eventType=TABLE_MAP, serverId=1745163349, headerLength=19, dataLength=1608, nextPosition=4837104, flags=0}
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:313) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:237) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:273) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more
Caused by: java.io.EOFException: Failed to read remaining 750 of 1314 bytes from position -624251094. Block length: 750. Initial block length: 1604.
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.fill(ByteArrayInputStream.java:115) ~[?:?]
	at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:105) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:48) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:313) ~[?:?]
	at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:237) ~[?:?]
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:273) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[?:?]
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[?:?]
	... 1 more



2026-05-18 00:22:57,380 INFO  [m.s.r.f.b.MySqlBinlogFetchTask] [debezium-reader-9] - Starting MySQL binlog reader,with timestamp filter 1778745540000
2026-05-18 00:22:57,380 INFO  [i.d.u.Threads                 ] [debezium-reader-9] - Requested thread factory for connector MySqlConnector, id = mysql_binlog_source named = binlog-client
2026-05-18 00:22:57,387 INFO  [i.d.u.Threads                 ] [debezium-reader-9] - Creating thread debezium-mysqlconnector-mysql_binlog_source-binlog-client
2026-05-18 00:22:57,388 INFO  [i.d.u.Threads                 ] [blc-xxx.xxx.com:3306] - Creating thread debezium-mysqlconnector-mysql_binlog_source-binlog-client
2026-05-18 00:22:57,393 INFO  [i.d.j.JdbcConnection          ] [pool-4164-thread-1] - Connection gracefully closed
2026-05-18 00:22:57,393 INFO  [m.s.r.f.b.MySqlBinlogFetchTask] [blc-xxx.xxx.com:3306] - Binlog client connected, closed idle jdbc connection.
2026-05-18 00:22:57,393 INFO  [i.d.u.Threads                 ] [blc-xxx.xxx.com:3306] - Creating thread debezium-mysqlconnector-mysql_binlog_source-binlog-client
2026-05-18 00:22:57,796 INFO  [m.s.r.f.b.MySqlBinlogFetchTask] [blc-xxx.xxx.com:3306] - skip binlog, currentTime:1778745356000, filterTime:1778745540000
2026-05-18 00:22:57,993 INFO  [m.s.r.f.b.MySqlBinlogFetchTask] [blc-xxx.xxx.com:3306] - skip binlog, currentTime:1778745366000, filterTime:1778745540000
2026-05-18 00:22:58,329 INFO  [m.s.r.f.b.MySqlBinlogFetchTask] [blc-xxx.xxx.com:3306] - skip binlog, currentTime:1778745376000, filterTime:1778745540000

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

java

Screenshots

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Contributor guide

Tech stack: javasql
Domain: backenddata
Issue type: bug
Difficulty: 4
Estimated time: 3-5 days
Activity status: fresh
Clarity: clear
Prerequisites: JavaMySQL CDC conceptsApache SeaTunnel
Newbie friendliness: 15
Research direction: Investigate the TIMESTAMP startup mode implementation in SeaTunnel's MySQL CDC connector. The issue is that when startup.mode=TIMESTAMP, the connector does not use the last committed offset from a checkpoint during recovery; instead it always reads from the configured startup.timestamp. This prevents recovery when binlogs are purged. To fix, review how the connector initializes the binlog position in MySqlStreamingChangeEventSource.java (likely using configuration parameter 'startup.timestamp' instead of stored offset). Examine the checkpoint related classes in the CDC source module to ensure the offset is restored. The fix should store the offset from the previous checkpoint and use it when recovering, falling back to startup.timestamp only on fresh runs. Consult the Debezium engine's offset management for guidance.