debezium/dbz

Create example connector [DBZ-1771]

开放

#285 创建于 2025年12月11日

 (6 条评论) (0 个反应) (0 位负责人)HTML (6 个派生)auto 404
component/documentationgood first issuemigrated-from-jiratype/task

仓库指标

星标
 (3 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

Migrated from DBZ-1771

There should be a very basic connector implementation, based on the new framework (as used by SQL Server, Postgres, etc.) that shows external contributors how to implement a CDC connector based on Debezium infrastructure.

The basic idea is to have a "database" represented by single file. Changes to that file will be tracked using Java's [WatchService|https://www.baeldung.com/java-nio2-watchservice]. The format of the file is like so:

{code} EventType;Id;FirstName;LastName;Salary;Active S|1|Alice|Smith|3000|true S|2|Bob|Miller|2000|false I|3|Bruce|Masters|4000|false U|1|Sarah|Smith|3000|false D|3|Bruce|Masters|4000|false EventType;Id;FirstName;LastName;Salary;Active;Birthday U|1|Sarah|Smith|3000|false|1980-12-31 ... {code}

Event type would be one of (S)napshot, (I)nsert, (U)date, (D)elete. I.e. this file is like a log, not like a table.

The snapshot change event source would read all the S events and then stop. The streaming source would read all the other rows and emit corresponding events (updates won't have the old value for sake of simplicity). Whenever the watch service detects a change of the file, the change event source will continue reading the file from the last processed offset (= line number) until the end.

The implementation should show:

  • snapshot source
  • streaming source
  • hand-over from the former to the latter, using the line number as offset
  • offset handling
  • value type conversion, based on the column types
  • Evolution of schema
  • Handling of a configuration option (the path and name of the database file)

Besides the implementation (which should really be as simple as possible), there should be accompanying documentation of the relevant pieces of the implementation.

贡献者指南