compare exit-code threshold (0.5) is an undocumented, untested magic number

Open Beginner friendly
#178 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Quiet
Tech stack
typescript
Domain
cli

Research direction

Start in packages/ghost/src/cli.ts and inspect both pairwise and --temporal exit paths, then review compare --help and the existing gate.test.ts and sync.test.ts coverage. Done means the shared 0.5 heuristic is named and documented, help text points serious CI users to --gate, and a test pins the cutoff behavior.

Written by the indexing model from the issue text.

Description

Summary

The pass/fail exit code for plain ghost compare uses a hardcoded distance > 0.5 cutoff, inlined twice in packages/ghost/src/cli.ts (the pairwise and --temporal paths). The value is undocumented, untested, and was introduced as a bare literal in the founding commit (618f275) with no rationale in the message or code.

Evidence
  • Inline literal at cli.ts (pairwise exit + temporal exit), no named constant, no comment.
  • No test pins the cutoff — gate.test.ts/sync.test.ts only use 0.5 as input data, not as a gate assertion. Changing the cutoff breaks zero tests.
  • The only place 0.5 is given meaning is docs/ideas/guided-migration.md (">0.5 = different design languages"), but that's an ideas doc and is not referenced by the code. The alignment is coincidental, not wired.
Why it matters

Users meet bare compare first and assume it's the drift gate. The real governance path is compare --gate (baseline-relative, per-dimension, acked via .ghost-sync.json). The arbitrary-looking 0.5 erodes trust in the number and obscures --gate.

Proposed fix (non-breaking)
  1. Promote 0.5 to a named, commented constant in cli.ts describing it as a heuristic and pointing to --gate for real gating.
  2. Add one line to compare --help steering serious CI users to --gate.
  3. (Optional, separate) consider a --threshold <n> flag only if there's real demand.
Related

The WEIGHTS constants in embedding/compare.ts have the same "taste captured once, undocumented" smell (1f8decb).

Dominant language
TypeScript
Stars
31
Forks
8
Avg merge
18h 15m
Merged PRs (30d)
21

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from block/ghost

All issues in block/ghost

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.