Table-model support for the Flink connectors: concrete design and three open questions
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
先建置並執行現有的兩個 Flink connector,然後閱讀 IoTDBSinkFunction 中所引用的路徑解析和 Session 呼叫附近的程式碼。比較對應的 Spark connector 目錄樹,並檢查 source、CDC 和 lookup 路徑,之後再決定獨立的 table connector 和 sink-first 實作是否合適。三個設計問題都有答案,且實作形態達成共識,就算完成。
由索引模型根據 Issue 內容生成。
描述
Concrete design: table-model support for the Flink connectors
Follow-up to [DISCUSS] Table-model support for the Flink connectors on
[email protected] (2026-08-08). That thread received no replies, so what
follows is the design I proposed there written out concretely. The three
questions I asked on the list are still open, and I have kept them open here
rather than treating silence as agreement on any of them.
The problem
flink-sql-iotdb-connector's schema mapping is the tree model, not a
configuration of it. In IoTDBSinkFunction a Flink column name is parsed as an
IoTDB path and split into a device and a measurement:
:132-136 PathUtils.splitPathToDetachedNodes(fieldName);
measurement = nodes[nodes.length - 1];
device = join(copyOfRange(nodes, 0, nodes.length - 1), '.');
:108-113 session.insertAlignedRecord(...) / session.insertRecord(...)
:86 new Session.Builder().nodeUrls(..).username(..).password(..).build()
In table mode there is no path to split. A column is a TAG, FIELD or ATTRIBUTE
under database.table, and which of the three it is carries meaning a name
cannot express. The connector's option list agrees that this is not a
configuration gap: there is no database and no dialect option, and aligned
and cdc.pattern are tree concepts.
Proposed shape
A new module flink-iotdb-table-connector, leaving flink-sql-iotdb-connector
untouched, mirroring how this repository already split Spark:
spark-iotdb-connector and spark-iotdb-table-connector are parallel trees with
their own parent poms, and the table one has its own spark-iotdb-table-common
rather than sharing the tree one's.
The objection to a separate module is duplicated CDC, lookup and bounded-scan
machinery. That objection applied equally to the Spark split and the project
accepted it there, so the cost is one this repository has already weighed for
this exact problem.
Three questions that are still open
The DISCUSS thread drew no replies. Lazy consensus covers "nobody objected to
the direction"; it does not answer these, and one of them rests on reading I
explicitly flagged as incomplete.
-
Is a separate
flink-iotdb-table-connectorthe right shape here? The
Spark precedent is the argument for it, but Spark's split may have had
reasons that do not carry over. -
Is the sink the right place to start? My reading was that the source's
tree couplings are the dialect-lessSessionand theTIMEclause — a
smaller and different problem from a mapping with no table-mode analogue —
which would put the design work in the sink. I have not read the CDC or
lookup paths. If the source has couplings I have not found, this ordering
is wrong and I would rather know before writing code than after. -
Is anyone already working on this? I searched issues and pull requests in
bothiotdb-extrasandiotdband found nothing on Flink and the table
model, but a search is not the same as asking.
What I plan to do next
Subject to the above: build and run both existing Flink connectors first. My
DISCUSS post was explicit that everything in it came from reading source and
that I had run neither connector. That is a reasonable basis for proposing a
shape; it is not a reasonable basis for implementing one, so running them is the
first step rather than a later one.
I am happy to take the implementation if the direction holds, and equally happy
to hand the design to whoever is better placed to do it.
- 主要語言
- Java
- 星號
- 6.4k
- 分支
- 1.2k
- 平均合併
- 1 天 11 小時
- 30 天內合併 PR
- 182
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
apache/iotdb 的其他 Issue
-
[Bug] sbin/replace-conf-from-env.sh word-splits the whole environment and corrupts iotdb-system.properties可能已有人在做 @anishmehta24 於 29 天前認領。 未關閉
難度 2/5 1-3 小時 新手友好度 82/100
apache/iotdb#18655 · 1 個 reaction ·
維護者通常 1 天內回覆
-
IoTDB Edge: stop-edge.sh does not stop its own process when IOTDB_HOME is set, and reports success可能已有人在做 @PDGGK 於 49 天前認領。 未關閉
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
[Bug] findColumn throws NullPointerException instead of SQLException for an unknown column name可能已有人在做 @PDGGK 於 90 天前認領。 未關閉
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
[Bug] ResultSetMetaData.getCatalogName throws IndexOutOfBoundsException instead of SQLException for out-of-range columns可能已有人在做 @PDGGK 於 92 天前認領。 未關閉
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 61/100
Vakore/ZappierGames#162 ·
-
enhancement
難度 2/5 1-3 小時 新手友好度 68/100
維護者通常 1 天內回覆
-
[Bug] Lock owned by another client (e.g. Text app) cannot be unlocked from the app可能已有人在做 @falkloewert 今天認領。 未關閉
難度 2/5 1-3 小時 新手友好度 72/100
維護者通常 1 天內回覆
-
bug grammar sev-0
難度 1/5 1 小時以內 新手友好度 86/100
-
難度 2/5 1-3 小時 新手友好度 76/100