Epic: converter convergence — one save-to-API contract, four implementations, no drift detection

Đang mở
#1,077 3 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
12/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
github-actions, javascript, node.js, python, typescript

Hướng nghiên cứu

Start with the Wave 1 work breakdown and its dependencies, then read comfy_cli/workflow_to_api.py, scripts/skew/run-cli.py, and src/converter/test/conversionParity.test.ts alongside the named pull requests. The first measurement step is to establish SKEW/1 with a declared catalog and comfy-cli revision; run the acceptance checks listed for each wave. Done means the converter paths have one reproducible parity measurement, the specified divergences and stale pin are addressed, and the roadmap's acceptance checks pass.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

agent-coded effort:large enhancement

Produced by a multi-agent investigation over the 500-fixture oracle corpus. Every number carries its measurement setting (see §2) because four incompatible "clean fixture" counts were already in circulation. Claims marked INFERRED were not read from code.

This issue IS the task decomposition — W1.1 through W4.6 each carry a repo, an effort tier, its dependencies, the fixtures it moves, and an acceptance check. Non-goals, open questions and the companion ADR draft are in the comments.

Related: comfy-cloud-mcp-server#1867 (the parity job's stale comfy-cli pin), comfy-cloud-mcp-server#1869 (PreviewAny over-emission), comfy-cli#893 (widget-type classification), #1070 (promoted subgraph values), #1071 (umask test fragility). Triage of the Comfy Hub conversion audit that started this: BE-24127.

Converter convergence roadmap

Scope: Comfy-Org/comfy-cli (comfy_cli/workflow_to_api.py, comfy_cli/cql/), Comfy-Org/comfy-cloud-mcp-server (src/converter/, which is also the source root of the published @comfyorg/api-to-workflow consumed by ComfyUI_frontend), and the oracle corpus at src/converter/test/fixtures/templates/. Comfy-Org/cql is tracked adjacently — see Non-goals.


1. The problem

One contract — "turn a saved workflow into the API prompt the backend executes" — has four independent implementations (frontend graphToPrompt(), @comfyorg/api-to-workflow, comfy-cli convert_ui_to_api, and cql's save-format slot writer), no shared code or data, and no mechanism that notices when they disagree. The user-visible cost is that the same workflow file executes differently depending on which path submitted it: comfy-cli currently converts 4 corpus fixtures to the wrong input image (a saved upload-button object aborts the whole schema walk, so LoadImage falls back to the combo's first option — comfy_cli/workflow_to_api.py:2389-2391); it drops user values entirely for CurveEditor.curve (issue #893, same line); it synthesizes optional widget values the user never set, so a pre-#1076 comfy run sends thinking=True and codec="none" on nodes the frontend sends nothing for; and slot_overrides on a subgraph instance writes to the wrong widget — 92.image lands in RandomNoise.noise_seed and silently changes the generated video (verified live against the pinned wasm). The reference that would catch all of this is frozen: the 500 goldens were captured 2026-09-11 against a frontend and an object_info nobody recorded, and they are now evaluated against an object_info that a cron rewrites daily — 142 of 500 fixtures (28.4%) fail for no reason but that the catalog moved under a fixed golden, which trains everyone to read real defects as staleness (and did: PreviewAny's three invented keys were waved through as "UI-only" for 52 fixtures while over-emitting on 148 of 148 goldens). The one job that compares two implementations pins comfy-cli to 231f85bf of 2026-07-05 — 139 commits and 114 days off main, and not an ancestor of main, so it cannot even detect that it is stale, and it insulates mcp CI from every comfy-cli fix and regression alike. Operationally: three sets of hand-maintained tolerance tables that nothing cross-checks, 4 tolerances that are provably dead (masking any future regression on those keys), a headline stat in KNOWN_ISSUES.md (249/455) that is stale in both numerator and denominator, and a corpus that already misses 38% of the shipped template index.


2. Measurement discipline (prerequisite for every number below)

Four different "clean fixture" counts circulate in the material behind this roadmap — package 266/500 and 394/500; comfy-cli 136/500 and 205/500 — because each was measured under a different combination of (tolerances on/off, catalog = current/at-capture, _meta in/out, comfy-cli revision pre/post #1076). None is wrong; they are not comparable. No item in this roadmap may be accepted against a number measured under an undeclared setting. Wave 1 therefore ships one measurement definition, and every subsequent acceptance check cites it.

Definition (SKEW/1): converter output vs expected_api.json; node-key sets equal; every input deep-equal after unwrapping {__value__} / {__type__,__value__}; _meta excluded both sides; node ids coerced to strings; catalog named explicitly (current | atCapture); tolerances off; comfy-cli revision recorded. Signature grammar: missing-input:<Class>.<in> · extra-input:<Class>.<in> · value:<Class>.<in> · missing-node:<id>:<Class> · extra-node:<id>:<Class> · class:<id>:<Expected>-><Actual>.


3. Corrections to assumptions this plan was nearly built on

Assumption Reality Consequence for the plan
comfy-cli #1069 makes the top-level widget walk follow saved inputs[].widget order It no longer does. Scope was cut after six reviewers objected that a consumed-slot count check cannot catch a permutation and that top-level inputs[] order is not a reliable record of widgets_values order. #1069 now reorders only COMFY_DYNAMICCOMBO_V3 sub-inputs (ByteDance2ReferenceNode, 10 of 392 Hub workflows). The "109 of 180 missing-input occurrences close for free" claim is void. SaveAudioMP3.audioUI (35), LoadAudio.audioUI (26), SaveGLB.image (34), Preview3D.image (14) need an explicit emission path. W3.1 is promoted from "free rider" to the largest single comfy-cli work item.
The marker reader does not exist yet #1076 already reads inputs[].widget markers to decide whether an optional widget was absent, and falls back to the default when there are no markers. W3.1 extends an existing reader rather than introducing one; the "emit undeclared marker names" decision is a one-line policy change at a site that already parses markers.
The _boundary_targets marker gate and the Load3DAdvanced layout split are unfiled Already open: #1039 (promoted value preserved at socket fan-out targets, fixes #974) is the marker-gate relaxation; #1056 shares the Load3DAdvanced legacy-layout check between edit and convert; #1057 strips the legacy button constants on edit; #1038 adds source_address to comfy workflow slots. Those move from "write" to "review and merge" in Wave 1/2. Do not open duplicates.
The stale pin means the MCP runs a 114-day-old converter COMFY_CLI_PIN is documented dead — nothing reads it at runtime (src/backend/cli-comfy-client.ts:27-33); live seams resolve comfy from COMFY_BIN/PATH. Harm is confined to (a) CI measuring a stale converter and (b) being the documented anchor for whoever packages the runtime image. Urgent, not an incident.

Wave 1 — already specified; start now

Nothing here depends on anything unbuilt. Three of six are open PRs awaiting review.

W1.1 — Merge comfy-cli #1072 (neutralize agent-attribution env vars in tests)

Repo comfy-cli · Tier trivial · Fixtures 0 (tooling) · Depends —
Change One autouse conftest.py fixture stripping CLAUDECODE / AI_AGENT / COMFY_USER_AGENT, sibling to the existing _neutralize_forced_color. Five test_code_search.py tests fail only when the suite is run from inside an AI tool.
Why first Every other comfy-cli item in this roadmap is verified by running the suite; until this lands, every contributor and agent sees 5 phantom failures and learns to ignore red.
Acceptance pytest tests/comfy_cli/command/test_code_search.py is 27/27 both with and without those vars exported; env -u run and plain run produce identical results.

W1.2 — Merge comfy-cli #1076 (stop synthesizing absent optional widgets)

Repo comfy-cli · Tier routine · Fixtures closes the largest cli-only cluster: CreateVideo.{bit_depth,codec,color_space} 116 each, SimpleMath+.b 111, ModelSamplingAuraFlow.sampling 97, CFGNorm.pre_cfg 43, SimpleMath+.a 20 · Depends W1.1 (to trust the suite)
Change Optional widget inputs absent from widgets_values are omitted; required inputs and optional non-widget sockets keep their schema default; socketless-required path untouched.
Acceptance The 5 signature families above drop to 0 occurrences under SKEW/1(catalog=atCapture); TestSeedreamDynamicCombo::test_pristine_pro_t2i_template_converts asserts thinking absent with the reasoning inline. Blocking caveat: the /tmp/skew differential was taken at comfy-cli c272a5e, which predates this PR — so its cli-side cluster counts are already stale and must be re-measured on post-merge main before any later item is sized against them.

W1.3 — Merge comfy-cli #1069 (dynamic-combo subs in saved order)

Repo comfy-cli · Tier routine · Fixtures 0 in the template corpus; 10 of 392 Hub workflows (Seedance 2.0 ByteDance2ReferenceNode) · Depends W1.1
Change Expands a selector's chosen option's sub-inputs in the order the save recorded, not the option's current order. Conservative in both directions (unknown sub name → current expansion; no dotted markers → unchanged; repeated name → first position; connection-only subs carried through).
Acceptance The 9 tests in test_workflow_to_api_saved_widget_names.py pass, 3 of which fail on main; on the live snapshot model.auto_upscale is absent and seed is the integer, not "randomize". Record explicitly in the PR that this does NOT change the top-level widget walk, so W3.1 is not written against a capability that does not exist.

W1.4 — mcp companion: ancestry guard + pin bump + KNOWN_DIVERGENCES rewrite

Repo par-mcp · Tier routine · Fixtures 3 of 4 KNOWN_DIVERGENCES entries deleted · Depends W1.2, W1.3 merged to comfy-cli main
Change One PR, three parts. (a) A step before the parity run: git -C .ci/comfy-cli fetch --depth=200 origin main && git merge-base --is-ancestor <pin> origin/main, failing with ci-bump-pin-guard.yml's existing wording — this fires today, the pin is NOT-ANCESTOR. (b) Advance all four 231f85bf literals (.github/workflows/ci.yml:401, src/converter/test/conversionParityCi.test.ts:24, src/backend/cli-comfy-client.ts:39, src/backend/cli-comfy-client.test.ts:408) to the post-#1076/#1069 SHA. (c) Delete the three padding entries from KNOWN_DIVERGENCES (conversionParity.test.ts:84+): template:03_video_wan2_2_14B_i2v_subgraphed, template:02_qwen_Image_edit_subgraphed, and the 127:78.sampling half of template:05_audio_ace_step_1_t2a_song_subgraphed — leaving only 59.audioUI (W3.1) and legacy_group_nodes.json.
Why coupled The suite asserts the divergence set bidirectionally — a resolved signature fails exactly like a new one — so #1076 landing will red this suite the moment the pin moves, and the stale pin is the only reason it has not already. Shipping (b) without (c) reds mcp main.
Acceptance npx vitest run src/converter/test/conversionParity.test.ts green with KNOWN_DIVERGENCES at 2 entries; the ancestry step fails on the old SHA and passes on the new one; cli-comfy-client.ts's comment block updated so the dead-constant caveat still reads true.

W1.5 — Batch the Python parity driver

Repo par-mcp · Tier routine · Fixtures 0 directly; unblocks all 500 in CI · Depends —
Change scripts/skew/run-cli.py: one process, stdin-driven loop, --templates <dir> --catalog <name>=<gz> (repeatable), emits {fixture: {catalog: apiGraph | {error}}}. Keep comfyCliConvert.py for single-fixture debugging (its fd-1→stderr dance at :70-85 is the right tool there).
Why Measured: 1.79 s per subprocess vs 6.8 ms of actual work — 224× overhead. 500 fixtures: ~15 min spawned, 5.2 s batched (0.42 s import + 0.32 s catalog inflate + 3.38 s convert). The 9-fixture corpus at conversionParity.test.ts:51-55 was never a corpus decision; it was the only affordable shape given execFileSync per fixture at :267-274.
Acceptance All 500 fixtures converted in one process in <15 s wall; output byte-identical to 500 individual comfyCliConvert.py invocations on a sampled 25.

W1.6 — Pin the oracle to its capture-epoch catalog

Repo par-mcp · Tier analytical · Fixtures +142 (266 → 408 package-side, SKEW/1) · Depends W1.7 (manifest) for the blob SHA; independent of W1.1-1.5
Change src/converter/test/fixtures/templates/MANIFEST.json carries objectInfoBlobSha: 2f37e189b847e7e9d5fbc08419e9e88d836aa767 (= a2c6d012:data/object_info.json.gz, 2026-09-11, md5 8caf766d…, 3655 classes). New src/converter/test/oracleCatalog.ts#loadEpochObjectInfo() materializes it via git cat-file blob <sha> into .ci/oracle-object-info/<sha>.json.gz (~700 KB, ~0.3 s, cached); on failure it falls back to data/object_info.json.gz and sets EPOCH_PIN_UNAVAILABLE, printed as ::warning:: — never a silent substitution. oracleFixtures.test.ts:84 switches to it. Then: delete isReparentedKey (:55-80, collapses to zero under the pin — the epoch snapshot declares SaveVideo.required = [video, filename_prefix, format, codec], bare); delete the 4 dead IGNORABLE_DIFFS entries (:13-14); keep LoadVideo.video-preview, Load3D.image, PreviewAny.* (they survive the pin, so they are not catalog drift); MIN_CORRECT 420 → 400 (stricter than 420 was, since the escape hatches are gone); MIN_WOULD_EXECUTE stays 498; re-enable the _meta.title test (:331) gated on the pin being available; STRUCTURAL_SKIP (:121-125) shrinks to flux1_dev_uso_reference_image_gen alone.
Acceptance Package clean count 408/500 under SKEW/1(catalog=atCapture); isReparentedKey deleted and the suite still green; all four Meshy/Quiver fixtures diff-free (verified by replay); EPOCH_PIN_UNAVAILABLE path exercised by a test that points the loader at a bogus SHA and asserts the warning is emitted and the fallback used.

W1.7 — Provenance manifest + capture validity gate

Repo par-mcp · Tier routine · Fixtures moves 2 from STRUCTURAL_SKIP to a named capture defect · Depends —
Change MANIFEST.json schema v1: capturedAt, capturedFrom, frontendVersion, frontendCommit, comfyuiVersion, objectInfoBlobSha, objectInfoCommit, templatesIndexCommit, captureParams, fixtureCount, captureErrors, degradedFixtures. Backfill from git (capturedAt = a2c6d012 author date 2026-09-11T17:50:40Z; frontendVersion: null, unrecoverable). export-oracle-fixtures.js reads window.__COMFYUI_FRONTEND_VERSION__ + GET /api/system_stats before the loop and emits a reserved __manifest__ bundle key; split-oracle-bundle.ts:48 routes that key to MANIFEST.json, not a fixture dir. Validity gate: after graphToPrompt() (:230), reject any result containing a node with falsy class_type or an input keyed UNKNOWN — push to _errors. Today template_animate_diff_loops and utility_face_detection_mediapipe carry 5 such nodes, captured from a browser session with packs unregistered, and were absorbed into STRUCTURAL_SKIP as converter known-issues.
Acceptance MANIFEST.json validates against its schema; a synthetic bundle containing a class_type-less node makes the splitter exit non-zero; the two degraded fixtures appear in degradedFixtures and no longer in STRUCTURAL_SKIP.


Wave 2 — closes measured gaps; depends only on Wave 1

W2.1 — Delete the package's PreviewAny registry entry

Repo par-mcp · Tier trivial · Fixtures +47 (394 → 441 under tolerances-on; 148 of 148 PreviewAny nodes over-emitted today) · Depends W1.6 (so the measurement is against a pinned catalog)
Change FRONTEND_EXTRA_WIDGETS.PreviewAny (widgetMapper.ts:40-44) emits three keys graphToPrompt() never emits — textPreviewWidgets.ts:39,:53,:66-67 sets options.serialize = false on both widgets it injects, and one of the three names (preview_markdown) does not exist at all. Replace with slot-consuming non-emitting entries on the PointsEditor shape: [{name:'preview_text',serialize:'none',requireSlot:true},{name:'previewMode',serialize:'none',requireSlot:true}]. Both flags are load-bearing: requireSlot is what stops the else if (!extra.requireSlot) inputs[extra.name] = '' fallback at :1576 from emitting anyway. Then delete the PreviewAny tolerance from both mirrored IGNORABLE_DIFFS (oracleFixtures.test.ts:14-18, scripts/oracle-fixtures/verify-oracle-fixtures.ts:83-90) so the suite measures it, and fix the source comment at widgetMapper.ts:24 — it cites src/extensions/core/previewAny.ts, a file that no longer exists.
Acceptance 0 extra-input:PreviewAny.* signatures over 500 fixtures; clean count +47; grep -rn PreviewAny src/converter/test/ scripts/ returns no tolerance entry. Before merging, run git log -S PreviewAny -- src/converter/widgetMapper.ts and cite in the PR which fixture (if any) motivated the entry, so the deletion cites evidence rather than only the current corpus.

W2.2 — comfy-cli: stop aborting the schema walk on a dict slot

Repo comfy-cli · Tier analytical · Fixtures 4 converted to the wrong input image today; plus CurveEditor.curve (#893) · Depends W1.1, W1.2
Change _collect_widget_inputs (workflow_to_api.py:2389-2391) currently does if any(isinstance(v, dict) for v in widget_values): _absorb_dict_widget_values(...); return out — abandoning the whole walk. Replace with a per-slot rule inside the walk: a dict slot whose options.serialize is False (or type == "button") is a frontend-injected non-serializing widget — consume the slot, emit nothing. Keep _absorb_dict_widget_values (:2432-2449) only for the lora / no-schema paths it was written for, entered only when the dicts are self-describing (type or lora keys) and the class declares no positional widget order. Highest-severity item in the whole roadmap: a saved upload-button object makes LoadImage lose its real filename.
Acceptance A fixture-level test asserting the 4 affected fixtures emit the saved filename, not the combo's first option; CurveEditor.curve round-trips its {__type__:"CURVE",__value__:…} value; no converter output ever contains an input literally named button or upload (see W2.5).

W2.3 — comfy-cli: promoted host-value fallback + merge #1039

Repo comfy-cli · Tier analytical · Fixtures 5 improved, 0 regressed, 11 signatures removed, 0 added (measured over all 500, same catalog) · Depends W1.1
Change Two lines on the existing path, not the proposed spike. (a) _overlay_promoted_host_values (workflow_to_api.py:285-287) does continue when the host value is UNSET; call comfy-cli's own already-correct frontend port instead — promoted.effective_value_for(...) / source_value (cql/promoted.py:815-847, :849-861). (b) Relax _boundary_targets' marker gate (promoted.py:2345-2353): when a target's serialized input entry has no widget marker, still accept it if entry["name"] is a declared input of the target class, or the class is unknown to object_info — the frontend inlines a promoted widget's value into every consumer of the boundary link (markerless INT,FLOAT, wildcard *, COMFY_MATCHTYPE_V3 all receive it in the goldens). This is what open PR #1039 does (fixes #974) — review and merge it rather than reimplementing; (a) is the piece that makes it reachable.
Explicitly do NOT wire flush_proxy_migration into convert_ui_to_api: measured over every proxyWidgets instance in all 500 fixtures, it creates 0-1 inputs per workflow, changes no output, and removes none of the 9 cluster signatures. #1070's blocking spike is testing a hypothesis that is already falsified.
Acceptance 5 named fixtures improve, byte-identical on the other 495, 0 signatures added; 1527 passed / 2 skipped across cql/ + workflow_to_api + command/workflow; a perf guard asserting conversion never constructs a full Graph (monkeypatch Graph.from_object_info to raise) — the eager Graph is 0.53 s vs 0.0029 s of conversion, ~180×; use a lazy catalog view (ctor 1.6 µs; warming all 591 distinct classes across all 500 fixtures costs 0.079 s total).
Add to the parity allowlist (conversionParity.test.ts:51-55, today 3 subgraph fixtures, none of this shape): gsc_starter_2 (nested legacy chain, value on a leaf two levels down), video-wan21_scail (markerless INT,FLOAT socket), video_ltx2_3_ic_lora (COMFY_MATCHTYPE_V3 socket).

W2.4 — comfy-cli: fix the write-side widgets_values_named mirror

Repo comfy-cli · Tier trivial · Fixtures 0 (0 of 500 fixtures carry the field) but a reproduced live defect · Depends W1.1
Change cql/engine.py:5113 gates the named-map mirror on has_group, so a plain set_widget on a non-dynamic node leaves the map stale. Reproduced: comfy-cli writes a file where it converts steps=30 while @comfyorg/api-to-workflow and the frontend both convert the same file to the pre-edit steps=20 — comfy-cli silently publishes a self-contradicting workflow whose edit the other two discard. Replace the gate with a module-level _mirror_named_widget_value(node, name, value) that fires whenever the key exists and is a dict, updating an existing key only (never adding one) — the exact rule of the package's mirrorNamedWidgetValue (widgetMapper.ts:2089-2106). Call it from the two other writers that skip it (engine.py:5270 same-option in-place dynamic-combo selector write, :5546 legacy-primitive widgets_values[0]). Move the shape guard at :4950-4951 off the has_group gate.
Acceptance A test that edits steps on a non-dynamic node carrying widgets_values_named, then asserts comfy-cli's and @comfyorg/api-to-workflow's conversions of the written file agree; grep -n has_group comfy_cli/cql/engine.py shows no remaining gate on a named-map mirror.

W2.5 — Regression guards for the two negative cases both sides drifted into

Repo both · Tier trivial · Fixtures guards, not fixes · Depends W2.1, W2.2
Change Assert over the 500-fixture corpus that (a) no converter output ever contains an input named upload, and (b) PreviewAny emits only source. Both are provable against the goldens and both are exactly where the two registries currently disagree with them.
Acceptance Each guard fails when the corresponding registry entry is reintroduced (prove it by reverting W2.1 locally and watching it red).

W2.6 — Rewrite KNOWN_ISSUES.md from the metric, not by hand

Repo par-mcp · Tier trivial · Depends W1.6, W2.1
Change The headline (:3-4) claims 249/455 (55%) against a 500-fixture corpus whose own test comment records 469/500 while the floor sits at 420. Replace with the SKEW/1 output, stating the catalog it was measured against. Close the TextGenerate / LTXVAudioVideoMask / GeminiImage2Node entries (§3, :49-59) as resolved-by-catalog-refresh — all are live classes (10, 1 and 42 fixtures respectively) whose divergence no longer fires. Correct §4: its stated cause for LoadVideo.video-preview (slot count) is measurably false — object_info declares one widget and all 44 saved nodes have identical two-slot widgets_values; document it as capture non-determinism (21 goldens carry it, 23 with identical inputs do not).
Acceptance Every number in the file is reproducible by one command; a test asserts that any IGNORABLE_DIFFS entry producing no signature on a class present in the corpus is a hard failure (3 lines, and the one self-healing property the current design lacks).


Wave 3 — closes the remaining measured gaps; needs Wave 1-2 infrastructure

W3.1 — comfy-cli: build the trailing frontend-injected-widget emission path

Repo comfy-cli · Tier deep · Fixtures closes 109 of 180 missing-input occurrences from the marker-derivable group alone; measured 136 → 160 on the pre-#1076 baseline · Depends W1.2 (#1076's marker reader), W1.3, W2.2, and no longer on #1069 (see §3)
Change comfy-cli has the consumption half and a partial name model (cql/engine.py:1011-1040 frontend_extra_widget_names, :167 FRONTEND_MARKER_SLOTS, :72-80 _LOAD_3D_BUTTON_SLOTS, :978-1008 load_3d_button_slots, pinned for the catalog by tests/comfy_cli/cql/test_frontend_widget_slots.py) but never emits — _schema_widget_pairs (workflow_to_api.py:1908) stops walking at the last object_info-declared input. Four parts, in order:

  1. After the ordered loop, walk the inputs[].widget marker names that follow the last declared input and emit each from its slot. Two value rules: a marker with no remaining slot still emits "" (SaveAudioMP3.audioUI, 35 goldens); a marker in the non-serializing set {upload} emits nothing while consuming its slot. Policy decision to record: emit undeclared marker names rather than intersecting with object_info — this is where the 109 occurrences live.
  2. Emit the Load3D button slots instead of skipping them (:1994-2003 currently does while widget_values[vidx] in LOAD_3D_BUTTON_VALUES: vidx += 1); append (name, value) from _LOAD_3D_BUTTON_SLOTS positionally. Goldens emit them as upload 3d model / upload extra resources / clear. Coordinate with #1056/#1057, which already touch this code.
  3. LoadAudio.audioUI computed value: saved slot is always null; emitted value is /api/view?filename={urlquote(inputs['audio'])}&type=input&subfolder=& (trailing & included), applied post-walk only when audioUI was emitted and audio is a non-empty string (mirrors widgetMapper.ts:1588-1595).
  4. A small registry for the entries markers do not name: ImageBatchMulti."Update inputs" (copy raw slot; golden value is null), PointsEditor.{editor_display → consume+emit nothing, "Reset canvas" → copy slot}, FL_PathAnimator.{"Edit Paths","Path Count"}, plus FL_PathAnimator's order override (paths_data is slot 0 at runtime but last in inputs[].widget, so no saved-order walk fixes it — express as the explicit prefix ["paths_data","*"]), plus a variadic rule for CustomCombo: after choice, emit index, then option{n} for every remaining slot, n from 1 (count = len(widgets_values) - 2, verified 3..9 across the corpus). _schema_widget_pairs must start reporting slots consumed (today the count lives in its internal vidx).
    Deliberately not added: PreviewAny (the package's entry over-emits on 148/148) and LoadVideo.video-preview (undecidable 21/23 corpus split — that belongs in the harness tolerance list, not converter data).
    Acceptance Per-cluster, measured after each sub-part lands, not once at the end: SaveAudioMP3.audioUI 35→0, LoadAudio.audioUI 26→0, SaveGLB.image 34→0, Preview3D.image 14→0, ImageBatchMulti."Update inputs" 18→0, Load3D buttons→0, CustomCombo.option{n} complete for N=3..9. The per-cluster arithmetic assumes a fixture's divergences are independent, and slot alignment is exactly where that breaks (emitting a previously-missing slot can shift a value that currently matches by accident) — so the gate is the total clean count, re-measured each time.
W3.2 — Fix CustomCombo on the package side (the one real both-sides defect)

Repo par-mcp · Tier analytical · Fixtures the only genuine both-sides defect of the 8 families examined · Depends W1.6
Change ExtraWidgetDef (widgetMapper.ts:33-38) needs a repeat flag; CustomCombo becomes [{name:'index',serialize:'value'},{name:'option',serialize:'value',repeat:true}], replacing the option1..option4 cap at :63-69 which under-emits option5-9. Three call sites read the list by fixed length and all three need the flag: the forward walk's for (const extra of extraWidgets) (:1560-1583), the slot-count/defaults helper's extraCount = registryEntry(...)?.length ?? 0 (:766-772), and the reverse api→workflow emitter (:2428-2450). index may keep reading slot 1 (saved == recomputed in all 10 golden instances) but recomputing it as options.indexOf(choice) matches serializeValue more faithfully.
Acceptance All 10 CustomCombo golden instances exact, N from 3 to 9; the reverse direction round-trips a 9-option node.

W3.3 — Widget-type classification: census + conformance vectors

Repo both · Tier analytical · Fixtures 46-odd inputs neither corpus can adjudicate; COLORCODE is latent-but-real, proven by template_purz_nb2_single_image_sprite_sheet · Depends W1.6, W3.1 (injected widgets are simultaneously a classification fact and an order fact)
Change The allowlist-membership framing is mostly a red herring: of 405 distinct input type strings in the 3853-class snapshot, 388 are handled by neither classifier and 370 of those are genuine link types where default-deny is correct. The real backlog is ~20 types, split three ways, and the classifiers disagree on rules, not membership: comfy-cli has widgetType and defaultInput rules the package lacks (43 inputs misclassified as links, incl. Preview3D.model_file); the package has a comma-split rule and CUSTOM_WIDGET_TYPES carrying COLORCODE, which comfy-cli misses on all 16 of its inputs. Ship two committed artifacts and two test files:

  • widget_type_census.json: type → {verdict: widget|link|hidden-only, registeredBy, declaredInputs, classes, evidence}. Generated, not authored: useWidgetStore().widgets is literally new Map([...customWidgets, ...Object.entries(ComfyWidgets)]) and inputIsWidget is widgets.has(type) (widgetStore.ts:20-27) — that map is the authoritative set. (Settles one open question already: MARKDOWN/CHART/GALLERIA/RANGE/PAINTER/TEXTAREA are not dead — VERIFIED present at src/scripts/widgets.ts:249-261.)
  • classifier-vectors.json: the ordered rule procedure as executable {type, options} → verdict cases plus a declared order (forceInput/defaultInput demotion → widgetType override → registered-type membership → socketless promotion → comma-split union → COMFY_*COMBO* family → wildcard refusal), each case carrying a failsToday: [impl] ledger that must shrink and is never a tolerance.
  • tests/comfy_cli/cql/test_widget_type_census.py and src/converter/test/widgetTypeCensus.test.ts, reading the same JSON. Three assertions: COVERAGE (an unknown type fails with its usage count and three example Class.input sites, so a new object_info drops a red test rather than a silent link), AGREEMENT (_is_widget_input and engine._is_link both match the census verdict — this is what catches the NUMBER split and the COLORCODE gap without a fixture), REACHABILITY (every allowlist member either has a declared input in the snapshot or is tagged injected: true with the injecting class list — this surfaces IMAGEUPLOAD/AUDIOUPLOAD/PREVIEW_3D, which are injected-by-name and therefore unreachable from _is_widget_input at all).
    Plus the cheap comfy-cli fixes the census pins: accept membership in engine._FRONTEND_REGISTERED_WIDGET_TYPES in _is_widget_input after the DOM-widget check, guarded behind the existing forceInput check (one line; picks up COLORCODE's 16 inputs and the 3 non-socketless COLORs); reconcile _IMPLICIT_WIDGET_TYPES (engine.py:36, has NUMBER) with the converter's inline primitive set (workflow_to_api.py:1838, has COMFY_DYNAMICGROUP_V3, lacks NUMBER) into one exported definition — today the engine calls Number Operation.number_a a widget port and the converter calls it a link, so set-widget can address a slot the converter will not write; add COLOR/COLORCODE/IMAGECOMPARE to _FRONTEND_DOM_WIDGET_TYPES (engine.py:48-59); delete the dead type_id != "*" conjunct at engine.py:1067 ("*" is not a member of _IMPLICIT_WIDGET_TYPES, so the guard can never fire); give the required-socketless fallback a type-keyed placeholder table instead of None (port getSocketlessDefault, widgetMapper.ts:433-449) but only when a slot exists and is null — SAM3_Detect.bboxes and SDPoseKeypointExtractor.bboxes have no slot and the golden omits the key entirely (#1076 fixed this for optional widget inputs; required-socketless is a separate branch and still over-emits).
    Acceptance The census test is non-vacuous: a poison fixture builds a synthetic object_info with one class CensusPoison carrying poison_widget: ["POISON_WIDGET",{}] and poison_link: ["POISON_LINK",{}], neither in the census, and asserts coverage raises; a second poison case asserts that a census entry claiming POISON_WIDGET is a widget while _is_widget_input says link fails the AGREEMENT check. Without both, a refactor that empties the type set or swallows the exception passes silently. Plus: COLORCODE widget on all 16 inputs in both implementations; NUMBER identical in engine and converter.
W3.4 — Read-side widgets_values_named (layered, no walk change)

Repo comfy-cli · Tier routine · Fixtures 0 of 500 (latent); closes one verified read-path divergence · Depends W2.4, W3.1
Change After pairs = _schema_widget_pairs(...) returns (workflow_to_api.py:2400-2406), test the node's map for completeness over exactly those pairs — for every (name, value) in pairs with value is not None, the map must carry a non-None own key name — and on pass, substitute named[name] for each pair's value while keeping the pair list (names, order, count) untouched. Because the walk is untouched this cannot left-shift a slot and needs no relaxation of the exact accounting check. Also: when widgets_values is absent but a usable map is present, drive the existing walk with an empty value list instead of returning {} (today: package {seed:7,steps:33} vs cli {seed:0,steps:20}). Deliberately do not port the package's "a named name is emitted even with no slot" extension — on a post-#15924 save the two representations have identical cardinality, so it is unreachable for frontend-written files, and adding it would let the map decide which widgets exist.
Acceptance The four exclusions asserted as tests, not code: a dict/keyed widgets_values ignores the map; _fallback_widget_names (:2086-2119) ignores it (else canvas-only widgets like control_after_generate reach the prompt under real names); a linked input's value comes from the link; promoted subgraph host values stay positional and never consult a leaked/stale map on the container node (promoted.py:901-931). Mirror the package's leak tests (namedWidgetValues.test.ts:661,:751). Measurement caveat: 0 of 500 fixtures and 0 of 11025 nodes carry this field, so the oracle suite is blind to all of it — this item ships against hand-authored fixtures (W4.4) or it ships untested.

W3.5 — cql slot addressing, closed in TypeScript (adjacent track)

Repo par-mcp · Tier analytical · Fixtures 0 (different contract); closes a live silent-corruption path · Depends —
Change On raw-templates/video_ltx2_i2v.json instance 92, 7 of 7 advertised slot addresses misresolve against the pinned wasm (cql_sha=37ab0b5d): 5 fail loudly on a -1 interior id, and 2 succeed silently — 92.ckpt_name → 11.control_after_generate (a no-op, since that widget is serialize: false, which is the ticket's headline exhibit and is not corruption) and 92.image → 11.noise_seed (an INT override that changes the generated video, which the ticket does not name). Extend preResolveTemplateOverrides (tools-saved-workflows.ts:1744) to pre-resolve declared slot addresses the way input_overrides already are (BE-9294), using the existing SubgraphInstanceIndex (template-overrides.ts:55-74): single remapTargets id → flat {"<instance>:<interior>": {...}}; containerProxiedNames (the -1 set) → write the host instance's widgets_values slot, which is what converter.ts:717 reads and what ADR-SUBGRAPH-PROMOTION-0009 mandates; ambiguous/unroutable → leave in slot_overrides and let the wasm answer. Reuse findProxyWidgetForDefSlot (groupNodeFlattener.ts:614) rather than writing a third resolver.
Do not fix resolveSlot's index in Go as the first move: cql implements the pre-ADR-0009 value model (follow proxyWidgets into the interior), which comfy-cli (promoted.py:17-19) and the mcp converter (converter.ts:710-719) have both abandoned in favour of host-wins — so fixing the index alone yields a correctly-wrong engine. File the Go fix (name-match → IO-link → refuse, never positional; ADR-0009 host semantics for -1) as an independent upstream cleanup, and merge BE-9295 into BE-7351 — they are the loud and silent faces of one assumption (positional correspondence between definition inputs[] and instance proxyWidgets), and BE-7351 already carries the 1124-of-2551-addresses sweep.
Acceptance src/cql-slot-addressing.test.ts pins the 7-row table as the pre-fix baseline against the real wasm and real converter (~15 s), then asserts all 7 land on their declared input post-fix, with 92.image → 11.noise_seed pinned explicitly; the ambiguous branch provably falls through to the wasm unchanged.


Wave 4 — prevents the next gap; depends on Waves 1-3

W4.1 — Generate the widget-layout table from the frontend's own browser tier

Repo ComfyUI_frontend + a new data bundle · Tier deep · Depends W3.1, W3.3
Change graphToPrompt()'s entire widget contract is 11 lines (executionUtil.ts:133-150: iterate node.widgets, skip !widget.name || widget.options.serialize === false, apply serializeValue, wrap arrays). The only unknown for a headless converter is therefore the content and order of node.widgets — a per-class fact, not an algorithm, currently approximated by hand in three places. A job that dumps it already exists: browser_tests/fixtures/customNode/allNodesMountTier.ts:93 reads (node.widgets ?? []) with name/type/options for every node in the manifest in batches of 24, and :223 already serializes widgetNames. Adding options.serialize and serializeValue provenance is ~30 lines in a tier that runs against a real backend with every manifest pack installed. Emit widget-layout.json: per class, an array of {slot, name, type, origin, promptEmit?, serializeValue?} plus an optional variadicTail. slot (index in a fresh node's widgets_values) subsumes order and pre-input injection, so PRE_INPUT_INJECTED_WIDGETS (widgetMapper.ts:83-95) and WIDGET_ORDER_OVERRIDES (:107-118) both disappear, as does comfy-cli's needed prefix list. promptEmit ∈ {value,none} is a verbatim read of options.serialize !== false. serializeValue has exactly three closed forms — const:<json>, template:<str with {urlencode:<input>} holes>, indexOf:<input> — covering the whole corpus; a fourth form is refused by the schema and goes in a named escapeHatches list instead. requireSlot is deliberately derived, not a field: identity needs a slot, a const does not, which is what makes SaveAudioMP3.audioUI=""-with-no-slot and PointsEditor.editor_display-consume-and-emit-nothing fall out of one rule instead of two flags.
Acceptance A class-coverage count that may never shrink between releases; a generated-by header check that rejects hand edits; the three existing package tables deleted with the suite still green; comfy-cli's vendored copy diffed against the published bundle by a test_widget_layout_vendored.py (mirroring src/converter/test/localCopies.test.ts:29-38), vendored rather than fetched so comfy workflow stays offline-safe (the precedent and the requirement are both cql/annotations_source.py:9-14, and pyproject.toml:91 already globs comfy_cli.cql.data for *.json).

W4.2 — Three-axis, two-catalog skew CI with a bidirectional manifest

Repo par-mcp (+ comfy-cli mirror) · Tier deep · Depends W1.5, W1.6, W4.1 is not required
Change Replace ci.yml's conversion-parity with a conformance job running all 500 fixtures in one batched pass across three axes (mcp-vs-golden, cli-vs-golden, mcp-vs-cli) × two catalogs (current, at-capture), gated by one skew-manifest.json checked bidirectionally — an extra signature and a resolved-but-still-listed signature both fail, generalizing KNOWN_DIVERGENCES, which already works that way at 9 fixtures. One row per signature (343 rows at today's measurement, vs 2386 for the per-fixture shape), each carrying axis, exact occurrences, catalogSensitive, verdict ∈ {defect, golden-stale, catalog-drift, undecidable}, owner, evidence, and a sunset required on every verdict except defect. catalogSensitive is machine-derived from the two-catalog run and then asserted against the stored value — a human cannot relabel a converter defect as staleness, which is the error this whole effort exists to prevent. Measured split today: 444 of 758 occurrences (58.6%) and 82 of 191 distinct signatures exist only because the catalog moved; at-capture-only is 0, so that bucket is a clean third alarm (converter now correct only against the new catalog → goldens unrepresentable → open a [recapture] issue, never block) rather than noise.
Delete the tautological tier: computePassingFixtures() (oracleFixtures.test.ts:130-166), the it.each(passingFixtures) block (:347-351), and critically the else { it.skip('no fixtures currently pass exact match') } vacuity branch (:352-354), which goes green when the converter breaks on all 500. Drop the '*' / structuralMarker hatch (:98-101, :362-374) — it existed only because the old grammar could not express node-set deltas, and extra-node:/missing-node:/class: now can. Keep the aggregate floors as a second tier beneath the manifest. Add the test-count floor the premise assumed already existed (grep -rn "PARITY_REQUIRED\|MIN_TESTS" returns nothing): vitest --reporter=json, then assert numTotalTests >= 500 && numPassedTests == numTotalTests && numPendingTests == 0 — today, if COMFY_CLI_PATH is unset or renamed, conversionParity.test.ts:307-313 takes it.skip before the non-empty-corpus assertion at :315-321, vitest exits 0, and test goes green with zero parity assertions executed. Exit codes 0/1/2 never collapsed ("we could not measure" must not read as "they diverged"), mirroring litegraph-parity.yml:66-91.
Acceptance 500 fixtures asserted, not a converter-selected subset; 266 fixtures assert the empty signature set; a _poison_skew fixture whose golden is deliberately wrong in three ways has those three signatures in the manifest and the suite reports them — a differ neutered to return empty sets reddens because the expected signatures go missing; budget ≈2 m 15 s (measured components: python 5.2 s + node 2.7 s + diff ~1 s, 3× runner derate).

W4.3 — parity-main.yml: advance the pin automatically, or freeze it visibly

Repo par-mcp · Tier routine · Depends W4.2
Change Daily 08:00 UTC (after refresh-object-info at 06:00), runs the identical skew:ci against comfy-cli main. Clean → open/force-update PR ci/bump-comfy-cli-pin advancing all four pin literals to the SHA just proven green (ci-bump-pin-guard.yml already gates provenance on that branch prefix). Red → open or update exactly one issue titled [skew] comfy-cli main drift (search-by-title dedup, so repeat days edit one issue rather than filing 114), body carrying the added/removed rows, their catalogSensitive labels, and the git log --oneline <pin>..<main> range so the suspect commit set is pre-narrowed; auto-close on the next clean run. Slack arm copied from router-spec-drift.yml:196-204 including its cancelled() branch (a timeout is the same silent-failure class). Never blocking.
Why this specific shape The bump PR cannot open while the issue is red, so ignoring the alert visibly freezes the pin. That inversion is the whole mechanism: today ignoring parity costs nothing, which is how 139 commits accumulated.
Acceptance Run it against the current main and confirm it files one issue, not 139; run it against the W1.4 SHA and confirm it opens a bump PR; run it twice red and confirm it edits one issue.
Risk to gate on: if parity-main's clean check is ever narrower than the blocking job's, the pin advances on weaker evidence than the gate demands. The two must share one script, differing only in the comfy-cli ref.

W4.4 — Hand-author the two shapes the corpus cannot supply

Repo both · Tier deep · Depends W3.4, W2.3
Change The oracle is blind to two things and will report green on both forever: widgets_values_named (0 of 500 fixtures carry it) and a legacy proxyWidgets instance with a populated host widgets_values (all 377 instances across 500 fixtures carry widgets_values: [], because templates ship at defaults by construction). Author both as language-neutral JSON + expected API, flagged provenance: "hand-authored" so their verdicts are never quoted as oracle-backed. The proxyWidgets fixture must include a null hole, a trailing short array, and a previewExposure/quarantine entry occupying a tuple index, so the tuple-index-vs-slot-index distinction is actually exercised; derive the expected output from the frontend's pickHostValue + applyHostValueToInput (proxyWidgetMigration.ts:194,:257-272,:507) and confirm once in a browser. The named-map set needs three variants (complete / partial / map-only) and ideally one post-#10392-pre-#15924 save, whose sparse positional holes are the only reason the hole exclusion exists.
Acceptance Both converters held to both fixtures; a test asserting that no hand-authored fixture is counted in any oracle-derived clean-count or floor.

W4.5 — Invert the golden capture into ComfyUI_frontend's CI

Repo ComfyUI_frontend + par-mcp · Tier deep · Depends W1.7 (manifest), W4.1 (same browser tier)
Change Today's corpus cannot be regenerated — cloud.comfy.org has moved — so a golden/converter disagreement is permanently unadjudicable, and scripts/oracle-fixtures/AGENTS.md:18-36 documents the live-browser path as needing a real authenticated session with a ~64-tab limit that fails silently partway through a long run. Propose an oracle-goldens job in the frontend repo: iterate the bundled template index, app.loadGraphData() + app.graphToPrompt() per template inside its existing browser-mode harness, upload oracle-goldens-<sha>.tar.zst (~3 MB compressed) plus the manifest with frontendCommit filled in by construction. Consume it from a new refresh-oracle-goldens.yml in par-mcp, a near-clone of refresh-object-info.yml: weekly cron, gh run download in place of the blob fetch at :178-187, reusing verbatim the no-diff clean exit (:262-296), the sha256 digest guard (:605-607), and the CLOUD_CODE_BOT app-token push so the auto-PR fires required CI.
Why it is the right shape, not just convenient It makes a golden reproducible per frontend commit instead of per wall-clock instant — bisectable rather than unadjudicable. The argument that gets the PR merged is that the frontend gains a graphToPrompt() regression guard it does not have; the PreviewAny cluster is exactly the frontend-side change that would have been caught at source. Headless, the 3-second settle becomes a deterministic flush of pending widget promises and the tab limit does not exist: ~2 min for ~730 templates vs 500 × 4.5 s ≈ 37 min of flaky serialized browser time.
Acceptance The gating spike first (see Open Questions Q13): one frontend browser-mode test that loads templates/default.json and asserts graphToPrompt().output deep-equals this corpus's templates-default/expected_api.json under a stubbed object_info. If app cannot boot headless, fall back to Playwright against the repo's own vite preview — still inside the frontend's CI, still no cross-org auth. The consuming cron must ::error:: on "no artifact in the last N runs" rather than exit clean — that exact failure mode is what produced the 139-commit pin rot.
Risk to gate on New goldens captured from a frontend that has itself regressed bake the regression in. The refresh PR body must print the per-fixture before/after delta, and any fixture moving clean→dirty or dirty→clean needs a one-line reason.

W4.6 — Grow the corpus to the current template index

Repo par-mcp · Tier routine · Depends W4.5
Change Against data/templates.json with prefix normalization: 277 of 727 index templates have no fixture (38%), and 50 fixtures name templates no longer in the index (api_openai_dall_e_2_t2i, api_stability_ai_*, api_veo2_i2v, the gsc_*/gsl_* set). The uncovered set is where all the new product is (image_flux3_klein_*, image_qwen_image_2_1_*, api_nano_banana_2_1_*, image_moss_image_*). Capture the missing 277; delete the 50 orphans with the reason recorded rather than skip-listing them (BE-11949 set that precedent). The 13 golden classes absent from the bundled catalog — OpenAIDalle2/3, Stability*, LtxvApi*, Kling*VideoEffectNode, VeoVideoGenerationNode — are withdrawn provider nodes, i.e. product churn that the catalog pin does not rescue; they need recapture or deletion, not a tolerance.
Acceptance coverageGap = uncovered / indexTemplates drops below 0.10; every absolute floor is re-expressed as a rate in the same PR that grows fixtureCount 500 → ~730 (never both an absolute re-baseline and a corpus growth in separate PRs); a staleness metric emitting goldenAgeDays, driftRate, coverageGap with a freshness SLA (recapture at goldenAgeDays > 45 ∨ driftRate > 0.10 ∨ coverageGap > 0.25 — all three breached today) and a hard build failure at goldenAgeDays > 90 ∨ driftRate > 0.35, so the corpus can never silently return to its current state.


Ngôn ngữ chính
Python
Star
1k
Fork
176
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
87

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của Comfy-Org/comfy-cli

Tất cả issue của Comfy-Org/comfy-cli

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.