Proof --intent checks a length limit on one part against the whole output
I maintainer di solito rispondono entro 5 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 43/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- bun, typescript
- Ambito
- developer-experience, testing
Direzione di ricerca
Start with countablesIn in skills/Assay/Tools/Proof.ts and compare its count checks with the judged asserts in IntentRubric.ts; the reproduction commands and shipped intents in the issue show the affected cases. Run both reproductions from the install root, then use the provided count script to inspect compiled checks. Done when checks measure the named output part or list item and the related requirements in a bullet are still checked.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
[!NOTE]
🤖 Avi (AI assistant) writing on behalf of Hans-Jakob. He read and approved this text before it was posted.
When an intent's Ideal State puts a length limit on one part of the output, such as "The Summary is one sentence of at most 25 words", Proof.ts run --intent measures that limit against the whole output, so a correct output fails on code.
Checked on LifeOS 8.761.0, with Proof.ts, IntentRubric.ts and every intent named below byte-identical to main at f6ab5e02, on Bun 1.3.13. Every command runs from the install root on shipped files and calls no model.
Reproduce
1. CreateSkill's own SKILL.md fails the CreateSkill intent
LIFEOS_GLANCE_DISABLED=1 LIFEOS_PROOF_RECEIPTS="$(mktemp -d)" \
bun skills/Assay/Tools/Proof.ts run \
--artifact skills/CreateSkill/SKILL.md \
--intent skills/CreateSkill/Intents/CreateSkill.md
[fail] intent-is-6: 8101 characters, above the maximum of 1024
Bullet 6 of that intent says the frontmatter description is "at most 1024 characters". The description in that file is 542 characters. The check measured the file, which is 8101 characters. LIFEOS_GLANCE_DISABLED=1 keeps the run on the code tier, so the judged checks read "unavailable"; that part is expected.
2. A short, correct summary fails the Fabric Summarize intent
Save this as summary.md:
ONE SENTENCE SUMMARY
Remote teams ship faster when decisions are written down before meetings instead of made inside them.
MAIN POINTS
- Written proposals let people respond on their own schedule, across time zones.
- Meetings shrink to the open questions the document could not settle.
- A decision log lets new hires see why past choices were made.
TAKEAWAYS
- Write the proposal first, then book the meeting only if questions remain.
- Keep one searchable log of decisions and their reasons.
LIFEOS_GLANCE_DISABLED=1 LIFEOS_PROOF_RECEIPTS="$(mktemp -d)" \
bun skills/Assay/Tools/Proof.ts run \
--artifact summary.md \
--intent skills/Fabric/Intents/Summarize.md
[fail] intent-is-3: 78 words, above the maximum of 20
[fail] intent-is-4: 78 words, above the maximum of 16
The one-sentence summary is 16 words and no main point is over 12. The intent puts "at most 20 words" on the one-sentence summary and "at most 16 words" on each main point. The two checks hold the whole summary to 16 words.
How many shipped intents this touches
This script compiles every intent and prints each count check it produces.
count-checks.ts
// Run from the install root: bun count-checks.ts
import { existsSync, readdirSync } from "node:fs";
const { loadIntent } = await import(`${process.cwd()}/skills/Assay/Tools/IntentRubric.ts`);
const { compileIntent } = await import(`${process.cwd()}/skills/Assay/Tools/Proof.ts`);
const root = process.argv[2] ?? "skills";
let intents = 0, withCount = 0, checks = 0;
for (const skill of readdirSync(root).sort()) {
const dir = `${root}/${skill}/Intents`;
if (!existsSync(dir)) continue;
for (const file of readdirSync(dir).filter((n) => n.endsWith(".md")).sort()) {
intents++;
const code = compileIntent(loadIntent(`${dir}/${file}`), "text").checks.filter((c) => c.type !== "glance");
if (code.length) withCount++;
for (const c of code) {
checks++;
console.log(`${skill}/${file} ${c.id} ${c.type} min=${c.min ?? "-"} max=${c.max ?? "-"}`);
}
}
}
console.log({ intents, withCount, checks });
On the intents of main it ends with { intents: 445, withCount: 53, checks: 74 }.
We read the bullet behind each of the 74 checks. By our reading:
| The number in the bullet is about | Checks |
|---|---|
| One named part, or each item in a list | 59 |
| A condition or the input, not a requirement on the output | 10 |
| The whole output | 2 |
| Could be read either way | 3 |
Some of the 69:
| Intent | The bullet says | What the check holds the output to |
|---|---|---|
| Fabric/Summarize | each main point is "at most 16 words" | at most 16 words in all |
| Fabric/RhetoricalLens | "under 250 words for a text under 50 words" | at most 49 words in all |
| Broadcast/Bluesky | "Text over 300 characters was cut to fit" | at least 301 characters |
| ISA/Seed | "L3 (one sentence, 60 words or fewer)" | at most 60 words in the whole ISA |
| StoryExplanation/CreateAbridged | five lines, "each 5-12 words" | 5 to 12 words in all |
| CreateSkill, UpdateSkill, ValidateSkill | the description is "at most 1024 characters" |
at most 1024 characters in the artifact |
All 74 checks with our reading
| Intent | Check | Compiled as | The number is about |
|---|---|---|---|
| Aphorisms/SearchAphorisms | intent-is-1 | characters, max 149 | condition: what the "short" filter means |
| Art/Aphorisms | intent-is-3 | words, 2 to 8 | either way: the quote on the image |
| BiasCheck/Check | intent-is-8 | sentences, 2 to 3 | part: the bottom line |
| BiasCheck/Check | intent-is-10 | words, 180 to 220 | condition: the length for one kind of source |
| Broadcast/Bluesky | intent-is-2 | characters, min 301 | condition: when text gets cut |
| CreateSkill/CreateSkill | intent-is-6 | characters, max 1024 | part: the frontmatter description |
| CreateSkill/OptimizeDescription | intent-is-5 | characters, max 1024 | either way: the description may be the whole artifact |
| CreateSkill/UpdateSkill | intent-is-7 | characters, max 1024 | part: the frontmatter description |
| CreateSkill/ValidateSkill | intent-is-2 | characters, max 1024 | part: the description being validated |
| ExtractWisdom/Alpha | intent-ac-3 | words, max 10 | part: each bullet |
| ExtractWisdom/Extract | intent-is-8 | words, 8 to 16 | part: each bullet |
| Fabric/AnalyzeBill | intent-is-3 | words, max 24 | part: the Summary |
| Fabric/AnalyzeBill | intent-is-7 | words, max 30 | part: the Verdict sentence |
| Fabric/AnalyzeClaims | intent-is-10-1 | words, max 30 | part: the Argument summary |
| Fabric/AnalyzeClaims | intent-is-10-2 | words, max 40 | part: the Overall analysis |
| Fabric/AnalyzeComments | intent-is-9 | words, max 15 | part: each point |
| Fabric/AnalyzeComments | intent-is-11 | words, max 20 | part: the Summary |
| Fabric/AnalyzeIncident | intent-is-9 | words, max 40 | part: the Summary sentence |
| Fabric/AnalyzeMalware | intent-is-3 | words, max 25 | part: the summary sentence |
| Fabric/AnalyzePaper | intent-is-3 | words, max 25 | part: the SUMMARY sentence |
| Fabric/AnalyzePaper | intent-is-6 | words, max 20 | part: the SUMMARY bullet in brief mode |
| Fabric/AnalyzeThreatReport | intent-is-3 | words, max 25 | part: the SUMMARY sentence |
| Fabric/CheckAgreement | intent-is-2 | words, max 30 | part: the Document summary |
| Fabric/CodingProject | intent-is-8 | words, max 20 | part: the PROJECT sentence |
| Fabric/CompareAndContrast | intent-is-7 | words, max 39 | part: each of two one-sentence lines |
| Fabric/CyberSummary | intent-is-3 | words, max 25 | part: the summary sentence |
| Fabric/CyberSummary | intent-ac-2 | words, max 25 | part: each item |
| Fabric/ExecutePattern | intent-ac-4 | characters, min 200 | condition: the input that triggers a harvest |
| Fabric/ExplainDocs | intent-is-4 | words, 23 to 28 | part: each of two sections |
| Fabric/ExtractPatterns | intent-is-7 | words, max 30 | part: the Analysis sentence |
| Fabric/ExtractReferences | intent-is-13-1 | words, max 16 | part: each bullet |
| Fabric/ExtractReferences | intent-is-13-2 | words, 8 to 12 | part: each bullet's usual length |
| Fabric/ExtractReferences | intent-ac-10 | words, max 16 | part: each bullet |
| Fabric/GitDiffSummary | intent-is-3 | characters, max 72 | part: the subject line |
| Fabric/NetworkThreatLandscape | intent-is-3 | words, max 120 | part: the Description |
| Fabric/NetworkThreatLandscape | intent-is-9 | words, max 24 | part: the One-Sentence Summary |
| Fabric/PullRequest | intent-is-2 | characters, max 72 | part: the title |
| Fabric/PullRequest | intent-is-4 | sentences, 2 to 4 | part: the Summary |
| Fabric/ReportFinding | intent-is-9 | words, max 24 | part: the One-Sentence Summary |
| Fabric/RhetoricalLens | intent-is-10-1 | words, max 249 | condition: the budget for one input size |
| Fabric/RhetoricalLens | intent-is-10-2 | words, max 49 | condition: the length of the input |
| Fabric/RhetoricalLens | intent-is-10-3 | words, max 299 | condition: the length of the input |
| Fabric/RhetoricalLens | intent-is-10-4 | words, a maximum (see the script output) | condition: the length of the input |
| Fabric/SecureByDesignQuestions | intent-is-2 | words, 23 to 28 | part: the overview paragraph |
| Fabric/SecureByDesignQuestions | intent-is-8-1 | words, 15 to 35 | part: the OVERVIEW |
| Fabric/SecureByDesignQuestions | intent-is-8-2 | words, 10 to 30 | part: each question |
| Fabric/SecurePrompt | intent-is-15 | words, a range around the stated target | whole: the whole deliverable |
| Fabric/SilentVictims | intent-is-11 | sentences, 1 to 4 | part: the OVERALL ASSESSMENT |
| Fabric/Summarize | intent-is-3 | words, max 20 | part: the ONE SENTENCE SUMMARY |
| Fabric/Summarize | intent-is-4 | words, max 16 | part: each main point |
| Fabric/SummarizeDebate | intent-is-2 | words, max 25 | part: the Summary |
| Fabric/SummarizeDebate | intent-is-3 | words, max 24 | part: the Primary disagreement |
| Fabric/SummarizeDebate | intent-is-9 | sentences, max 3 | part: the Prediction |
| Fabric/SummarizeDebate | intent-is-10 | bullets, max 5 | part: the Follow-up |
| Fabric/SummarizeMeeting | intent-is-3 | words, max 25 | part: the OVERVIEW |
| Fabric/SummarizeMeeting | intent-ac-3 | words, max 25 | part: each item |
| Fabric/TerraformPlan | intent-is-14 | words, max 25 | part: the Summary |
| Fabric/ThreatScenarios | intent-is-13 | paragraphs, 1 to 3 | part: the NARRATIVE ANALYSIS |
| HyperFrames/CapabilityVideo | intent-is-3 | words, 40 to 75 | either way: the narration script |
| ISA/Seed | intent-is-2-1 | bullets, exactly 4 | part: level L2 of the ladder |
| ISA/Seed | intent-is-2-2 | words, max 60 | part: level L3 of the ladder |
| ISA/Upgrade | intent-is-3 | bullets, exactly 4 | part: level L2 of the ladder |
| RedTeam/TestIdea | intent-is-3 | sentences, 1 to 2 | part: the statement of the idea |
| Research/InterviewResearch | intent-is-2 | paragraphs, 2 to 3 | part: the COMPANY SUMMARY |
| StoryExplanation/Create | intent-is-1 | sentences, 5 to 15 | part: the body between the opening and closing sentence |
| StoryExplanation/CreateAbridged | intent-is-2 | words, 5 to 12 | part: each of five lines |
| StoryExplanation/CreateWithLinks | intent-is-2 | words, 15 to 25 | part: the opening sentence |
| SystemsThinking/FindArchetype | intent-is-1 | sentences, 2 to 4 | part: the behavior description |
| VideoEssay/Argue | intent-is-5 | words, the stated range | part: the section "The Argument" |
| VideoEssay/Explainer | intent-is-1 | words, 370 to 400 | condition: an example of a two-minute script |
| Vigil/Emit | intent-is-1 | characters, max 139 | part: the title |
| Visualize/Visualize | intent-is-5 | sentences, 2 to 5 | part: the Summary |
| Workflows/Reminder | intent-is-3 | characters, max 399 | whole: the reminder text is the artifact |
| X/Read | intent-is-2 | characters, min 281 | condition: which posts count as long |
Two related effects
- A bullet that yields a count check gets only that check in a run. Bullet 6 of the CreateSkill intent also asks for
version: 1.0.0, a short description,USE WHENand anevals:entry, and none of that is asked.intentRubrickeeps a judged assert for the rest of such a bullet; the run path does not. - Bullet 7 of the same intent ends "stays under 500 lines". Lines have no counter, so the bullet is dropped whole and its other requirements go unchecked too.
Where it comes from
countablesIn in skills/Assay/Tools/Proof.ts reads a count phrase as the output's own length unless notOutputLength finds a cue beside it: a position word ("the first 150 characters"), "each", "every" or "per" before it, a hyphenated or singular unit in front of another noun ("the 1300-character fold"), or an example inside a comparison. The comment above that function already says what should happen: a property of one part never becomes a length check, and the bullet goes to the judge.
The shipped intents usually name the part as the subject of the sentence: "The Summary is …", "OVERVIEW is at most 25 words", "L3 (one sentence, 60 words or fewer)". Nothing reads the subject, so the default applies. Conditions such as "Text over 300 characters was cut" read as requirements for the same reason.
That default suits an ask such as "write this in under 80 words". An Ideal State describes a deliverable with several named parts, and there most count phrases are about a part.
Two ways it could go
The choice is yours. We have not written a patch, and will send one for whichever you prefer.
- In
compileIntent, make a count phrase a code check only when its clause says it is about the whole output: "total", "overall", "whole", or a subject such as "the output" or "the response". Every other count bullet stays a judged question. It is a small change, and part limits are then judged instead of counted. - Let a bullet name the part it bounds, a heading or a frontmatter key, and measure only that part. That is more work, and it gives real code checks for section lengths.
- Lingua principale
- TypeScript
- Stelle
- 19.9k
- Fork
- 2.5k
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di danielmiessler/LifeOS
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 67/100
danielmiessler/LifeOS#2894 ·
I maintainer di solito rispondono entro 5 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
danielmiessler/LifeOS#2891 ·
I maintainer di solito rispondono entro 5 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
danielmiessler/LifeOS#2888 ·
I maintainer di solito rispondono entro 5 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
danielmiessler/LifeOS#2886 ·
I maintainer di solito rispondono entro 5 giorni
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 68/100
danielmiessler/LifeOS#2848 ·
I maintainer di solito rispondono entro 5 giorni
Tutte le issue di danielmiessler/LifeOS
Issue simili
-
ready-for-agent
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 86/100
LucasSantana-Dev/Lucky#3257 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
XiaomiMiMo/MiMo-Code#2849 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement frontend good first issue help wanted testing-infrastructure
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 2 giorni
-
enhancement follow-up
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
obsidian-tasks-group/obsidian-tasks#4155 ·
I maintainer di solito rispondono entro 1 giorno