debezium/dbz

Create example connector [DBZ-1771]

開放

#285 建立於 2025年12月11日

 (6 則留言) (0 個反應) (0 位負責人)HTML (6 個分叉)auto 404
component/documentationgood first issuemigrated-from-jiratype/task

倉庫指標

星標
 (3 顆星)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

Migrated from DBZ-1771

There should be a very basic connector implementation, based on the new framework (as used by SQL Server, Postgres, etc.) that shows external contributors how to implement a CDC connector based on Debezium infrastructure.

The basic idea is to have a "database" represented by single file. Changes to that file will be tracked using Java's [WatchService|https://www.baeldung.com/java-nio2-watchservice]. The format of the file is like so:

{code} EventType;Id;FirstName;LastName;Salary;Active S|1|Alice|Smith|3000|true S|2|Bob|Miller|2000|false I|3|Bruce|Masters|4000|false U|1|Sarah|Smith|3000|false D|3|Bruce|Masters|4000|false EventType;Id;FirstName;LastName;Salary;Active;Birthday U|1|Sarah|Smith|3000|false|1980-12-31 ... {code}

Event type would be one of (S)napshot, (I)nsert, (U)date, (D)elete. I.e. this file is like a log, not like a table.

The snapshot change event source would read all the S events and then stop. The streaming source would read all the other rows and emit corresponding events (updates won't have the old value for sake of simplicity). Whenever the watch service detects a change of the file, the change event source will continue reading the file from the last processed offset (= line number) until the end.

The implementation should show:

  • snapshot source
  • streaming source
  • hand-over from the former to the latter, using the line number as offset
  • offset handling
  • value type conversion, based on the column types
  • Evolution of schema
  • Handling of a configuration option (the path and name of the database file)

Besides the implementation (which should really be as simple as possible), there should be accompanying documentation of the relevant pieces of the implementation.

貢獻者指南