Watched lessons watch too little: fix each miss §4af found, and drop files from the three lessons CI runs

Open
  • Is something wrong?
#318 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@emeraldleaf is already working on this.

Since Nov 1, 2026.

  • #328 by @emeraldleaf — open

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
50/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
github-actions, python
Domain
devtools, testing-qa

Research direction

Start from the receipt drift-mutate-20261031-1522.json on #312's branch to see the 22 misses. For coverage, run okl update <id> --files '' for three lessons and widen governed files like src/okl/client.py and docs/. For sensitivity, edit lesson checks to cover all records and rewordings. Done when §4af's re-run reaches 0.5 catch rate.

Written by the indexing model from the issue text.

Description

enhancement

REPORT §4af broke each replayable watched lesson on purpose. The receipt is drift-mutate-20261031-1522.json, on #312's branch.

Drift caught 14 of 36 judged breaks (38.9%, Wilson 24.8–55.1%). Of the 22 misses:

  • 14 changed a file the lesson does not govern.
  • 8 were red, but the lesson's check passed.

Two follow-ups were registered to fire on this result, and both do.

1. Three lessons lose their governed files (#39)

For n_71c09abf3f37 (lint-imports), n_b74bad29fd87 and n_be9f13d587b1 (pytest selections), CI failed on every break drift caught (5 of 5). Drift adds nothing for them. By the repo's own rule (#39), clear their files with okl update <id> --files ''.

2. Each miss goes to its cause

Coverage misses. Widen the governed files to what the claim depends on (the store's watch-what-the-rule-depends-on):

  • doc-check-defaults: add src/okl/client.py and src/okl/service.py. Two breaks changed defaults there.
  • docs-agent-neutral: its claim covers every doc, but it governs only README.md and docs/GETTING-STARTED.md. Three breaks were in docs/COMMANDS.md, docs/COSTS-AND-KNOBS.md and docs/RETRIEVAL.md. Govern docs/ and widen the check to match. This overlaps #314.
  • n_d45d7bf64896 (no superiority claims): two breaks were in docs/posts/ and docs/RELATED-WORK.md. Govern every public doc.
  • n_e635fd921086 (the package name): one break was in packaging/org-knowledge-layer/pyproject.toml.
  • n_7e50e3588bcd: a break in pyproject.toml made an import undeclared. Govern pyproject.toml too.

Sensitivity misses. Widen each check to the claim (#189):

  • n_07faa4565128, all four breaks: the check reads one record in one seed file, while the breaks were in other records of the same kind. Check every record the rule covers.
  • n_d45d7bf64896: a fixed phrase list missed a reworded claim.
  • docs-agent-neutral: a heading grep missed a rewording.

3. Then measure again

Re-run §4af's instrument on these lessons after the fixes. The registered bar for describing drift as detection is a catch rate of 0.5. Until a re-run reaches it, the docs describe drift as bookkeeping, a re-check prompt; #312 makes that wording change.

Before widening, count the cost: wider file lists mean more re-checks. §4ae's replay can count those re-checks on the history without running any new check.

Dominant language
Python
Stars
1
Forks
1
Avg merge
6h 2m
Merged PRs (30d)
108

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from emeraldleaf/okl

All issues in emeraldleaf/okl

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.