Doc claims are checked in the README and guide only; the rest of the docs drift unwatched

Open
  • Is something wrong?
#314 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
documentation

Research direction

Start from ci/check-init-flags.py and the doc-claim-governs-doc-and-code pattern in the store. Read the verification-scope-is-the-thesis context to understand lesson coverage. List claim shapes across docs/, CHANGELOG.md, and docstrings, then generalize the flag gate to all commands and add lessons for unhandled shapes. Done when a flag-and-command gate fails on a broken flag, claim lessons exist, and a before/after coverage receipt is committed.

Written by the indexing model from the issue text.

Description

enhancement

Problem

The README promises that documentation is "proven by a check you choose, flagged when what it covers changes". Today four doc-claim lessons watch two files, the README and docs/GETTING-STARTED.md:

  • doc-check-defaults
  • doc-init-flags
  • doc-install-name
  • doc-verified-refused

No lesson governs claims made anywhere else:

  • the rest of docs/;
  • CHANGELOG.md;
  • docstrings;
  • evals/REPORT.md, apart from the figures test_docs.py checks.

Separate gates catch some shapes: diagram figures that cite receipts, links, orphaned docs, and tombstones (#221).

On 2026-10-31, review found three stale claims; okl found none of them:

  • §4ad said the backend contract and the conformance suite check the tokenizer case on both backends. They do not.
  • v2.set_stem's docstring said the product stems by default. It has not since §4ad.
  • The CHANGELOG called a conformal 5% null rate "guaranteed". The guarantee holds only on average.

All three sat in places no lesson governs.

Proposal

  1. List the claim shapes. Find the shapes docs use that code can contradict:
    • a command or flag;
    • a default;
    • a number;
    • a file path;
    • a statement about what the code does or tests.
  2. Use a gate for a repeating shape. For example, every okl <command> --flag written anywhere in the docs exists in the parser. This generalises ci/check-init-flags.py from okl init to every command.
  3. Use one lesson for each remaining claim. It governs both the doc and the code it describes, with a check that fails when they disagree. This is the pattern in the store's doc-claim-governs-doc-and-code.
  4. Report coverage. Count the share of tracked docs that at least one claim check or gate covers, before and after.

Acceptance

  • A flag-and-command gate over every tracked doc, proven red by breaking one flag on purpose.
  • Claim lessons for the shapes the gate cannot cover.
  • A coverage figure, before and after, with a committed receipt.

Context: verification-scope-is-the-thesis in the store. The thesis covers both directions, and this issue is the repo-drifts-from-its-docs half.

Dominant language
Python
Stars
1
Forks
1
Avg merge
5h 22m
Merged PRs (30d)
119

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from emeraldleaf/okl

All issues in emeraldleaf/okl

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.