Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

[Feature] Add query-scoped planning limits and cooperative cancellation

オープン
  • 何か問題がありますか？
#435 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 3 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
8/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
活発
技術スタック
cpp
領域
data-engineering

調査の方向性

This is design work, so start by reading the API surfaces in include/paimon/scan_context.h and include/paimon/table/source/table_scan.h, then the limit ordering in src/paimon/core/table/source/data_table_batch_scan.cpp and the manifest fan-out in src/paimon/core/operation/file_store_scan.cpp (ReadFileEntries). The first step is agreeing the API shape with maintainers on this thread. Done means an approved contract plus tests that inject allocation, I/O and submission failures.

索引モデルが issue の本文から書いたものです。

説明

Parent integration: sirius-db/sirius#2181. Consumer bridge tracking: polardb/duckdb-paimon#139.

Need

A database embedding paimon-cpp needs to admit planning work before materializing an arbitrarily large native plan, and stop submitting work after cooperative cancellation. This is a feature request, not a claim that existing standalone planning is incorrect.

The integration pins 53f9c86d45aabb0a6f1a379271da07d7a9f27a3d. The following API gap also remains in inspected main 242decffdfdefef01dcd0e52e48c9a1e69792479:

  • ScanContextBuilder::WithMemoryPool routes pool-backed allocations, but does not bound ordinary STL planning containers or their transitive metadata allocations.
  • TableScan::CreatePlan() returns a complete plan; it exposes no planning cancellation/budget contract. SetLimit is a row-result pushdown, not a planning-memory quota: DataTableBatchScan::CreatePlan calls starting_scanner_->Scan before ApplyPushDownLimit.
  • FileStoreScan::ReadFileEntries builds a vector of futures for all supplied manifest metadata, then collects complete per-manifest entry vectors. A custom serial executor can prevent concurrent task execution, but does not itself bound those retained containers/results.

Source references at the inspected main:

Proposed Contract To Discuss

Provide optional query-scoped planning controls, preserving existing unlimited behavior by default. The API shape should be agreed with maintainers; a post-plan split-count check is not sufficient.

  • Check conservative manifest/entry/split/retained-metadata quotas before the corresponding allocation or task submission; identify which allocations are covered and what remains outside the guarantee.
  • Bound outstanding manifest work and retained results, including ordinary containers, rather than only pool-backed buffers.
  • Check a cooperative cancellation token before I/O/submission and inside substantial decode/merge loops. Do not promise interruption of an arbitrary blocking filesystem call unless the filesystem supports it.
  • Return a distinguishable resource-refusal/cancellation status, never partial success or EOF. Stop new submissions and drain accepted work before returning, including exceptions and submission failure.
  • Avoid changing process-global executors, Arrow thread capacity, or allocation resources.

Acceptance tests should use many small manifests and injected allocation/I/O/submission failures. Assert the limit is enforced before excess work is accepted, accepted jobs are drained on every terminal path, and cancellation cannot return a partial successful plan. Include serial/nested executor configurations to guard against deadlock. Keep row LIMIT semantics separate.

This issue does not request another Arrow batch-lifetime implementation; that work already exists in #259/#260. No hard whole-process-memory bound or prompt cancellation of uninterruptible I/O is claimed by the current integration.

主要言語
C++
スター
68
フォーク
34
平均マージ
4日 37分
マージ済み PR（30日）
35

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

apache/paimon-cpp のほかの issue

apache/paimon-cpp の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。