Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

[Feature] Add query-scoped planning limits and cooperative cancellation

未关闭
  • 哪里不对？
#435 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 3 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
8/100
Issue 类型
功能
描述清晰度
需要澄清
活跃度
活跃
技术栈
cpp
领域
data-engineering

调研方向

This is design work, so start by reading the API surfaces in include/paimon/scan_context.h and include/paimon/table/source/table_scan.h, then the limit ordering in src/paimon/core/table/source/data_table_batch_scan.cpp and the manifest fan-out in src/paimon/core/operation/file_store_scan.cpp (ReadFileEntries). The first step is agreeing the API shape with maintainers on this thread. Done means an approved contract plus tests that inject allocation, I/O and submission failures.

由索引模型根据 Issue 内容生成。

描述

Parent integration: sirius-db/sirius#2181. Consumer bridge tracking: polardb/duckdb-paimon#139.

Need

A database embedding paimon-cpp needs to admit planning work before materializing an arbitrarily large native plan, and stop submitting work after cooperative cancellation. This is a feature request, not a claim that existing standalone planning is incorrect.

The integration pins 53f9c86d45aabb0a6f1a379271da07d7a9f27a3d. The following API gap also remains in inspected main 242decffdfdefef01dcd0e52e48c9a1e69792479:

  • ScanContextBuilder::WithMemoryPool routes pool-backed allocations, but does not bound ordinary STL planning containers or their transitive metadata allocations.
  • TableScan::CreatePlan() returns a complete plan; it exposes no planning cancellation/budget contract. SetLimit is a row-result pushdown, not a planning-memory quota: DataTableBatchScan::CreatePlan calls starting_scanner_->Scan before ApplyPushDownLimit.
  • FileStoreScan::ReadFileEntries builds a vector of futures for all supplied manifest metadata, then collects complete per-manifest entry vectors. A custom serial executor can prevent concurrent task execution, but does not itself bound those retained containers/results.

Source references at the inspected main:

Proposed Contract To Discuss

Provide optional query-scoped planning controls, preserving existing unlimited behavior by default. The API shape should be agreed with maintainers; a post-plan split-count check is not sufficient.

  • Check conservative manifest/entry/split/retained-metadata quotas before the corresponding allocation or task submission; identify which allocations are covered and what remains outside the guarantee.
  • Bound outstanding manifest work and retained results, including ordinary containers, rather than only pool-backed buffers.
  • Check a cooperative cancellation token before I/O/submission and inside substantial decode/merge loops. Do not promise interruption of an arbitrary blocking filesystem call unless the filesystem supports it.
  • Return a distinguishable resource-refusal/cancellation status, never partial success or EOF. Stop new submissions and drain accepted work before returning, including exceptions and submission failure.
  • Avoid changing process-global executors, Arrow thread capacity, or allocation resources.

Acceptance tests should use many small manifests and injected allocation/I/O/submission failures. Assert the limit is enforced before excess work is accepted, accepted jobs are drained on every terminal path, and cancellation cannot return a partial successful plan. Include serial/nested executor configurations to guard against deadlock. Keep row LIMIT semantics separate.

This issue does not request another Arrow batch-lifetime implementation; that work already exists in #259/#260. No hard whole-process-memory bound or prompt cancellation of uninterruptible I/O is claimed by the current integration.

主要语言
C++
星标
68
派生
34
平均合并
4 天 37 分钟
30 天内合并 PR
35

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

apache/paimon-cpp 的其他 Issue

查看 apache/paimon-cpp 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。