Expose terminal failed-column and seed-row provenance in public results
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
- #861 @andreatnvidia による — マージされずにクローズ
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 45/100
調査の方向性
packages/data-designer/src/data_designer/interface/results.py の DatasetCreationResults から始め、async_scheduler.py、task_model.py、dataset_builder.py、data_designer.py を通じて終端失敗の処理を追跡します。seed のアイデンティティと現在の実行における失敗レコードが create()、preview()、型付き例外を通じてどのように流れるかを定義します。async_trace やログの解析を必要とせずに、公開結果コントラクトが記載された受け入れ例を満たせば完了です。
索引モデルが issue の本文から書いたものです。
説明
Priority Level
High (Major improvement)
Is your feature request related to a problem? Please describe.
Consumers that compose many columns into one Data Designer graph cannot determine, from the public run result, which column terminally failed for each omitted seed row.
Data Designer internally knows the task column, row group, and row index when a task fails. DatasetCreationResults.task_traces can expose those fields when full async tracing is enabled, but that is a heavy per-task diagnostic mechanism and the row position is not a stable caller-provided record identity. On a partially successful run, a consumer can diff input and output IDs to discover that a row disappeared, but cannot attribute that loss to the originating column without parsing logs.
This blocks an integration in NVIDIA-NeMo/Anonymizer#237 and PR #240. Anonymizer's legacy rewrite implementation uses several Data Designer runs, so a missing row can be attributed to a stage such as replacement-map generation, initial rewrite, evaluation, or repair. The combined conditional graph uses one Data Designer run. Without column-level failure provenance, every omitted row can only be reported as failing the coarse rewrite-combined boundary.
The required contract is a stable mapping from each terminally omitted seed row to its originating failed column, plus a safe failure category or reason. It must be available without enabling full task tracing.
Describe the solution you'd like
Expose lightweight, structured terminal row failures through the public result API for create() and preview(). If a run raises instead of returning a result, make the same provenance available on the typed exception.
The exact class and field names are flexible, but consumers need this information for each terminal failure:
@dataclass(frozen=True)
class TerminalRowFailure:
seed_record_id: str | int
column: str
failure_kind: str
reason: str | None = None
The public API should also provide a way for the caller to designate a seed identity column, for example _anonymizer_record_id, whose value is copied into seed_record_id. The returned identity must be directly joinable to the original input and must not require reconstructing it from (row_group, row_index). If no identity column is configured, Data Designer may expose a documented stable seed-row locator as a fallback.
Required semantics:
- Capture terminal row omissions after retry and salvage behavior has completed.
- Attribute the root column that failed, not downstream columns that were skipped or cancelled because their dependency was missing.
- Do not report
skip.whendecisions as failures. - Do not report transient failures that later recovered.
- Work for LLM, custom, expression, and other row-producing column types that can omit a row.
- Remain available when only part of a dataset succeeds and when every row fails.
- Be collected independently of
async_trace; do not retain prompts, completions, row contents, or successful task traces. - Document whether failure records cover the current invocation or the full resumed run.
- Preserve deterministic input ordering where practical.
A result shape such as DatasetCreationResults.failed_records and the equivalent preview result field would satisfy the need. A lazy public artifact referenced by the result would also work if it provides the same typed, current-run contract without requiring log parsing.
Acceptance examples:
- Given two seed rows with designated IDs, if column
evaluate_0terminally fails for one row, the result contains exactly that seed ID andcolumn="evaluate_0". - If downstream
repair_0is not scheduled becauseevaluate_0failed, no separaterepair_0failure is reported. - If
repair_0is skipped becauseskip.whenis true, no failure is reported. - If a retry succeeds, no terminal failure is reported.
- The same information is available without
async_trace=Trueand for an all-rows-failed typed exception.
Describe alternatives you've considered
- Keep the Anonymizer pipeline split across multiple Data Designer runs. This preserves coarse stage attribution but retains custom orchestration and prevents a single conditional graph from becoming the default.
- Diff input and output record IDs. This identifies omitted rows but not the column that caused the omission.
- Enable
async_trace=Trueand map(row_group, row_index)back to the seed. This retains every task trace, is not appropriate as a scalable production result API, and does not expose the caller's stable record ID. - Parse warning logs, or consume proposed structured log records from #386. Logs and telemetry are useful diagnostics, but they are not a reliable post-run outcome contract and can be filtered or missed by handlers.
- Preserve failed fields as
nullas proposed in #362. That changes row-retention semantics; this request only asks Data Designer to report the provenance of its existing terminal row omissions.
Agent Investigation
Investigation of current main found:
packages/data-designer/src/data_designer/interface/results.py:DatasetCreationResultsexposestask_traces, but no terminal failure collection.packages/data-designer-engine/src/data_designer/engine/dataset_builders/scheduling/task_model.py:TaskTracecontainscolumn,row_group,row_index,status, anderror, but traces are only created when tracing is enabled and do not contain a caller-provided seed identity.packages/data-designer-engine/src/data_designer/engine/dataset_builders/async_scheduler.py: the scheduler has the originatingTaskwhen recording terminal failures and already distinguishes failures, retries, and skips internally.packages/data-designer-engine/src/data_designer/engine/dataset_builders/dataset_builder.py: the builder exposes only the first non-retryable exception for the special zero-record error path, not per-row terminal failure provenance.packages/data-designer/src/data_designer/interface/data_designer.py: task traces are copied toDatasetCreationResults, while terminal failure records are not returned.
Existing issues #386 and #362 are adjacent but do not provide a typed result mapping terminally omitted rows to both the originating column and the caller's seed ID.
Additional context
This is not a request for full tracing, model-call telemetry, or preservation of failed row contents. The intended surface is a lightweight correctness and failure-attribution contract for applications that embed Data Designer graphs.
Checklist
- I've reviewed existing issues and the documentation
- This is a design proposal, not a "please build this" request
- 主要言語
- Python
- スター
- 2.4k
- フォーク
- 234
- 平均マージ
- 23時間
- マージ済み PR(30日)
- 75
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
NVIDIA-NeMo/DataDesigner のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 85/100
NVIDIA-NeMo/DataDesigner#1089 ·
メンテナーはふだん 1 日以内に返信
-
`ValidationColumnConfig` misaligns rows and discards the whole batch after an upstream row is dropped対応中かも @suryapilla が 2 日前に担当しました。 オープンtriaged
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
NVIDIA-NeMo/DataDesigner#1073 · コメント 4 件 ·
メンテナーはふだん 1 日以内に返信
-
docs: required_columns description is incomplete for LLM and multimodal columns対応中かも @rksharma-owg が 9 日前に担当しました。 オープンbug triaged
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
NVIDIA-NeMo/DataDesigner#1002 · コメント 4 件 ·
メンテナーはふだん 1 日以内に返信
-
enforce \from future import annotations` via ruff FA102 rule`対応中かも @chethanuk が 44 日前に担当しました。 オープンtask
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
NVIDIA-NeMo/DataDesigner#760 ·
メンテナーはふだん 1 日以内に返信
-
難易度 3/5 1〜2日 初心者へのやさしさ 55/100
NVIDIA-NeMo/DataDesigner#1088 ·
メンテナーはふだん 1 日以内に返信
NVIDIA-NeMo/DataDesigner の issue をすべて見る
似ている issue
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 84/100
GoogleCloudPlatform/gce-rescue#195 ·
メンテナーはふだん 2 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 83/100
メンテナーはふだん 1 日以内に返信
-
intake mcp-intake needs-ac needs-human-review priority:medium
難易度 2/5 1〜3時間 初心者へのやさしさ 67/100
Ikalus1988/MisakaNet#3848 · コメント 2 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 75/100
521xueweihan/HelloGitHub#4105 ·
-
bug size:XS
難易度 2/5 1〜3時間 初心者へのやさしさ 83/100
メンテナーはふだん 1 日以内に返信