Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

[Feature] Split the table hierarchy and materialize the store a table is served by

Đang mở
  • Có gì không ổn?
#459 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 3 ngày

@lucasfang đang làm issue này rồi.

Từ ngày 20/10/2026.

  • #447 của @lucasfang — đang mở

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
20/100
Loại issue
Tái cấu trúc
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
cpp
Lĩnh vực
backend, databases

Hướng nghiên cứu

The issue describes changes across Table, FileStoreTable, FormatTable, SystemTable, RestCatalog::GetTable, and the table scan/read paths, but names no source files or tests. Start by locating those definitions and the existing table hierarchy, then read the open pull request #447 to understand the work already underway. Done means the hierarchy and file system behavior match the described solution and the affected tests pass.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

A single Table class dispatched everything: the differences between managed, format, and system tables lived in if branches inside one 1400-line table.cpp, every operator call (NewScan/NewRead/NewWrite/NewCommit) re-merged a fresh options map and re-parsed CoreOptions — the state Java Paimon's AbstractFileStore parses exactly once — and the file system a table is served by was resolved before the schema's access configuration was even known.

The ambiguity produced real defects:

  • RestCatalog::GetTable(Identifier("sys", "tables")) returned the read-only schema of a global system table but still passed it to the managed-table load, which statically cast it to TableSchema and called PrimaryKeys() on it — undefined behavior.
  • The operator factory path built the file system from the context options alone; a schema carrying remote backend and credentials (delivered through SetTableSchema()) was therefore served by the default local backend, and reading its data failed.
  • The $ro and $audit_log views built their underlying scans and reads from the context's file system, ignoring the one their table was opened with; a context that also carried a file-system scheme map was then refused outright (WithFileSystem() and WithFileSystemSchemeToIdentifierMap() cannot be used together).
  • FileStoreTable::Copy half-applied a branch option: it was written into the copy's schema options but never moved the copy's branch, so the consistency check reported the branch actually served as a conflict and let a mismatched one through.
  • An option conflict between a context and a held table was reported with both raw values in the error message, leaking credentials into logs.
  • With REST data tokens enabled, the system tables a GetTable() call returned were bound to the catalog-wide file system, so $snapshots and friends read the data table's files without the table-level credentials and failed.

Solution

  • Split the hierarchy: Table becomes an abstract base (NewXxx pure virtual, a new virtual Copy(dynamic_options)), implemented by FileStoreTable — with AppendOnlyFileStoreTable/PrimaryKeyFileStoreTable subclasses — by FormatTable (which now derives from Table and owns its own assembly, absorbing format_table_loader), and by the SystemTable family.
  • Introduce a FileStore (the counterpart of Java's AbstractFileStore) that materializes CoreOptions once at construction, resolves the served file system after the options are known (schema-carried access configuration names its own backend; the caller's explicit file system wins over it), carries the file-system scheme map, and hosts the operator factories (NewPathFactory, NewSnapshotManager, NewManifestList, ...). FileStoreTable holds its schema/snapshot managers and its branch; SwitchToBranch derives a table of another branch.
  • Give context options one clear place to speak: the load. A factory-path load (TableScan::Create(ctx) etc.) merges the context options over the schema's through Table::Copy() — a table is the product of its load — while NewXxx on a held table refuses a contradicting option (option 'X' is 'A' in the context but 'B' on the table, values redacted). Copy never half-applies type or branch; managers are handed over unless a file-system option among the copy's changes the resolution, in which case fresh ones are built.
  • Fix the surrounding defects: RestCatalog::GetTable serves sys-database global system tables locally and binds per-table system tables to GetTableFileSystem(); $ro/$audit_log/$binlog prefer the context's file system and fall back to the table's only when the context names neither a file system nor a scheme map; the scan-fallback read context merges the load's dynamic options so the fallback reader recognises what the main one does.

Anything else?

  • Options a schema carries that do not parse now fail at table-build time rather than at the first operator; a table opened with neither a schema-carried backend nor a caller file system is served by the default local one instead of holding a null one.
  • No storage format or protocol change. Public API changes: Table is abstract (constructor protected, Name/FullName/NewXxx/Copy virtual, Table::CurrentBranch() replaced by FileStoreTable::CurrentBranch()), FormatTable derives from Table (static FormatTable::Copy(table, options) becomes the virtual Table::Copy(options), GetLocation() becomes GetRootPath()), and FileStoreTable::SwitchToBranch is new.

Are you willing to submit a PR?

  • I'm willing to submit a PR!
Ngôn ngữ chính
C++
Star
70
Fork
36
Merge trung bình
3 ngày 16 giờ
Pull request đã merge (30 ngày)
38

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của apache/paimon-cpp

Tất cả issue của apache/paimon-cpp

Issue tương tự

Thêm issue về C++

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.