Proof --intent checks a length limit on one part against the whole output
Maintainer thường phản hồi trong vòng 5 ngày
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức phù hợp với người mới
- 43/100
- Loại issue
- Lỗi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Công nghệ
- bun, typescript
- Lĩnh vực
- developer-experience, testing
Hướng nghiên cứu
Start with countablesIn in skills/Assay/Tools/Proof.ts and compare its count checks with the judged asserts in IntentRubric.ts; the reproduction commands and shipped intents in the issue show the affected cases. Run both reproductions from the install root, then use the provided count script to inspect compiled checks. Done when checks measure the named output part or list item and the related requirements in a bullet are still checked.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
[!NOTE]
🤖 Avi (AI assistant) writing on behalf of Hans-Jakob. He read and approved this text before it was posted.
When an intent's Ideal State puts a length limit on one part of the output, such as "The Summary is one sentence of at most 25 words", Proof.ts run --intent measures that limit against the whole output, so a correct output fails on code.
Checked on LifeOS 8.761.0, with Proof.ts, IntentRubric.ts and every intent named below byte-identical to main at f6ab5e02, on Bun 1.3.13. Every command runs from the install root on shipped files and calls no model.
Reproduce
1. CreateSkill's own SKILL.md fails the CreateSkill intent
LIFEOS_GLANCE_DISABLED=1 LIFEOS_PROOF_RECEIPTS="$(mktemp -d)" \
bun skills/Assay/Tools/Proof.ts run \
--artifact skills/CreateSkill/SKILL.md \
--intent skills/CreateSkill/Intents/CreateSkill.md
[fail] intent-is-6: 8101 characters, above the maximum of 1024
Bullet 6 of that intent says the frontmatter description is "at most 1024 characters". The description in that file is 542 characters. The check measured the file, which is 8101 characters. LIFEOS_GLANCE_DISABLED=1 keeps the run on the code tier, so the judged checks read "unavailable"; that part is expected.
2. A short, correct summary fails the Fabric Summarize intent
Save this as summary.md:
ONE SENTENCE SUMMARY
Remote teams ship faster when decisions are written down before meetings instead of made inside them.
MAIN POINTS
- Written proposals let people respond on their own schedule, across time zones.
- Meetings shrink to the open questions the document could not settle.
- A decision log lets new hires see why past choices were made.
TAKEAWAYS
- Write the proposal first, then book the meeting only if questions remain.
- Keep one searchable log of decisions and their reasons.
LIFEOS_GLANCE_DISABLED=1 LIFEOS_PROOF_RECEIPTS="$(mktemp -d)" \
bun skills/Assay/Tools/Proof.ts run \
--artifact summary.md \
--intent skills/Fabric/Intents/Summarize.md
[fail] intent-is-3: 78 words, above the maximum of 20
[fail] intent-is-4: 78 words, above the maximum of 16
The one-sentence summary is 16 words and no main point is over 12. The intent puts "at most 20 words" on the one-sentence summary and "at most 16 words" on each main point. The two checks hold the whole summary to 16 words.
How many shipped intents this touches
This script compiles every intent and prints each count check it produces.
count-checks.ts
// Run from the install root: bun count-checks.ts
import { existsSync, readdirSync } from "node:fs";
const { loadIntent } = await import(`${process.cwd()}/skills/Assay/Tools/IntentRubric.ts`);
const { compileIntent } = await import(`${process.cwd()}/skills/Assay/Tools/Proof.ts`);
const root = process.argv[2] ?? "skills";
let intents = 0, withCount = 0, checks = 0;
for (const skill of readdirSync(root).sort()) {
const dir = `${root}/${skill}/Intents`;
if (!existsSync(dir)) continue;
for (const file of readdirSync(dir).filter((n) => n.endsWith(".md")).sort()) {
intents++;
const code = compileIntent(loadIntent(`${dir}/${file}`), "text").checks.filter((c) => c.type !== "glance");
if (code.length) withCount++;
for (const c of code) {
checks++;
console.log(`${skill}/${file} ${c.id} ${c.type} min=${c.min ?? "-"} max=${c.max ?? "-"}`);
}
}
}
console.log({ intents, withCount, checks });
On the intents of main it ends with { intents: 445, withCount: 53, checks: 74 }.
We read the bullet behind each of the 74 checks. By our reading:
| The number in the bullet is about | Checks |
|---|---|
| One named part, or each item in a list | 59 |
| A condition or the input, not a requirement on the output | 10 |
| The whole output | 2 |
| Could be read either way | 3 |
Some of the 69:
| Intent | The bullet says | What the check holds the output to |
|---|---|---|
| Fabric/Summarize | each main point is "at most 16 words" | at most 16 words in all |
| Fabric/RhetoricalLens | "under 250 words for a text under 50 words" | at most 49 words in all |
| Broadcast/Bluesky | "Text over 300 characters was cut to fit" | at least 301 characters |
| ISA/Seed | "L3 (one sentence, 60 words or fewer)" | at most 60 words in the whole ISA |
| StoryExplanation/CreateAbridged | five lines, "each 5-12 words" | 5 to 12 words in all |
| CreateSkill, UpdateSkill, ValidateSkill | the description is "at most 1024 characters" |
at most 1024 characters in the artifact |
All 74 checks with our reading
| Intent | Check | Compiled as | The number is about |
|---|---|---|---|
| Aphorisms/SearchAphorisms | intent-is-1 | characters, max 149 | condition: what the "short" filter means |
| Art/Aphorisms | intent-is-3 | words, 2 to 8 | either way: the quote on the image |
| BiasCheck/Check | intent-is-8 | sentences, 2 to 3 | part: the bottom line |
| BiasCheck/Check | intent-is-10 | words, 180 to 220 | condition: the length for one kind of source |
| Broadcast/Bluesky | intent-is-2 | characters, min 301 | condition: when text gets cut |
| CreateSkill/CreateSkill | intent-is-6 | characters, max 1024 | part: the frontmatter description |
| CreateSkill/OptimizeDescription | intent-is-5 | characters, max 1024 | either way: the description may be the whole artifact |
| CreateSkill/UpdateSkill | intent-is-7 | characters, max 1024 | part: the frontmatter description |
| CreateSkill/ValidateSkill | intent-is-2 | characters, max 1024 | part: the description being validated |
| ExtractWisdom/Alpha | intent-ac-3 | words, max 10 | part: each bullet |
| ExtractWisdom/Extract | intent-is-8 | words, 8 to 16 | part: each bullet |
| Fabric/AnalyzeBill | intent-is-3 | words, max 24 | part: the Summary |
| Fabric/AnalyzeBill | intent-is-7 | words, max 30 | part: the Verdict sentence |
| Fabric/AnalyzeClaims | intent-is-10-1 | words, max 30 | part: the Argument summary |
| Fabric/AnalyzeClaims | intent-is-10-2 | words, max 40 | part: the Overall analysis |
| Fabric/AnalyzeComments | intent-is-9 | words, max 15 | part: each point |
| Fabric/AnalyzeComments | intent-is-11 | words, max 20 | part: the Summary |
| Fabric/AnalyzeIncident | intent-is-9 | words, max 40 | part: the Summary sentence |
| Fabric/AnalyzeMalware | intent-is-3 | words, max 25 | part: the summary sentence |
| Fabric/AnalyzePaper | intent-is-3 | words, max 25 | part: the SUMMARY sentence |
| Fabric/AnalyzePaper | intent-is-6 | words, max 20 | part: the SUMMARY bullet in brief mode |
| Fabric/AnalyzeThreatReport | intent-is-3 | words, max 25 | part: the SUMMARY sentence |
| Fabric/CheckAgreement | intent-is-2 | words, max 30 | part: the Document summary |
| Fabric/CodingProject | intent-is-8 | words, max 20 | part: the PROJECT sentence |
| Fabric/CompareAndContrast | intent-is-7 | words, max 39 | part: each of two one-sentence lines |
| Fabric/CyberSummary | intent-is-3 | words, max 25 | part: the summary sentence |
| Fabric/CyberSummary | intent-ac-2 | words, max 25 | part: each item |
| Fabric/ExecutePattern | intent-ac-4 | characters, min 200 | condition: the input that triggers a harvest |
| Fabric/ExplainDocs | intent-is-4 | words, 23 to 28 | part: each of two sections |
| Fabric/ExtractPatterns | intent-is-7 | words, max 30 | part: the Analysis sentence |
| Fabric/ExtractReferences | intent-is-13-1 | words, max 16 | part: each bullet |
| Fabric/ExtractReferences | intent-is-13-2 | words, 8 to 12 | part: each bullet's usual length |
| Fabric/ExtractReferences | intent-ac-10 | words, max 16 | part: each bullet |
| Fabric/GitDiffSummary | intent-is-3 | characters, max 72 | part: the subject line |
| Fabric/NetworkThreatLandscape | intent-is-3 | words, max 120 | part: the Description |
| Fabric/NetworkThreatLandscape | intent-is-9 | words, max 24 | part: the One-Sentence Summary |
| Fabric/PullRequest | intent-is-2 | characters, max 72 | part: the title |
| Fabric/PullRequest | intent-is-4 | sentences, 2 to 4 | part: the Summary |
| Fabric/ReportFinding | intent-is-9 | words, max 24 | part: the One-Sentence Summary |
| Fabric/RhetoricalLens | intent-is-10-1 | words, max 249 | condition: the budget for one input size |
| Fabric/RhetoricalLens | intent-is-10-2 | words, max 49 | condition: the length of the input |
| Fabric/RhetoricalLens | intent-is-10-3 | words, max 299 | condition: the length of the input |
| Fabric/RhetoricalLens | intent-is-10-4 | words, a maximum (see the script output) | condition: the length of the input |
| Fabric/SecureByDesignQuestions | intent-is-2 | words, 23 to 28 | part: the overview paragraph |
| Fabric/SecureByDesignQuestions | intent-is-8-1 | words, 15 to 35 | part: the OVERVIEW |
| Fabric/SecureByDesignQuestions | intent-is-8-2 | words, 10 to 30 | part: each question |
| Fabric/SecurePrompt | intent-is-15 | words, a range around the stated target | whole: the whole deliverable |
| Fabric/SilentVictims | intent-is-11 | sentences, 1 to 4 | part: the OVERALL ASSESSMENT |
| Fabric/Summarize | intent-is-3 | words, max 20 | part: the ONE SENTENCE SUMMARY |
| Fabric/Summarize | intent-is-4 | words, max 16 | part: each main point |
| Fabric/SummarizeDebate | intent-is-2 | words, max 25 | part: the Summary |
| Fabric/SummarizeDebate | intent-is-3 | words, max 24 | part: the Primary disagreement |
| Fabric/SummarizeDebate | intent-is-9 | sentences, max 3 | part: the Prediction |
| Fabric/SummarizeDebate | intent-is-10 | bullets, max 5 | part: the Follow-up |
| Fabric/SummarizeMeeting | intent-is-3 | words, max 25 | part: the OVERVIEW |
| Fabric/SummarizeMeeting | intent-ac-3 | words, max 25 | part: each item |
| Fabric/TerraformPlan | intent-is-14 | words, max 25 | part: the Summary |
| Fabric/ThreatScenarios | intent-is-13 | paragraphs, 1 to 3 | part: the NARRATIVE ANALYSIS |
| HyperFrames/CapabilityVideo | intent-is-3 | words, 40 to 75 | either way: the narration script |
| ISA/Seed | intent-is-2-1 | bullets, exactly 4 | part: level L2 of the ladder |
| ISA/Seed | intent-is-2-2 | words, max 60 | part: level L3 of the ladder |
| ISA/Upgrade | intent-is-3 | bullets, exactly 4 | part: level L2 of the ladder |
| RedTeam/TestIdea | intent-is-3 | sentences, 1 to 2 | part: the statement of the idea |
| Research/InterviewResearch | intent-is-2 | paragraphs, 2 to 3 | part: the COMPANY SUMMARY |
| StoryExplanation/Create | intent-is-1 | sentences, 5 to 15 | part: the body between the opening and closing sentence |
| StoryExplanation/CreateAbridged | intent-is-2 | words, 5 to 12 | part: each of five lines |
| StoryExplanation/CreateWithLinks | intent-is-2 | words, 15 to 25 | part: the opening sentence |
| SystemsThinking/FindArchetype | intent-is-1 | sentences, 2 to 4 | part: the behavior description |
| VideoEssay/Argue | intent-is-5 | words, the stated range | part: the section "The Argument" |
| VideoEssay/Explainer | intent-is-1 | words, 370 to 400 | condition: an example of a two-minute script |
| Vigil/Emit | intent-is-1 | characters, max 139 | part: the title |
| Visualize/Visualize | intent-is-5 | sentences, 2 to 5 | part: the Summary |
| Workflows/Reminder | intent-is-3 | characters, max 399 | whole: the reminder text is the artifact |
| X/Read | intent-is-2 | characters, min 281 | condition: which posts count as long |
Two related effects
- A bullet that yields a count check gets only that check in a run. Bullet 6 of the CreateSkill intent also asks for
version: 1.0.0, a short description,USE WHENand anevals:entry, and none of that is asked.intentRubrickeeps a judged assert for the rest of such a bullet; the run path does not. - Bullet 7 of the same intent ends "stays under 500 lines". Lines have no counter, so the bullet is dropped whole and its other requirements go unchecked too.
Where it comes from
countablesIn in skills/Assay/Tools/Proof.ts reads a count phrase as the output's own length unless notOutputLength finds a cue beside it: a position word ("the first 150 characters"), "each", "every" or "per" before it, a hyphenated or singular unit in front of another noun ("the 1300-character fold"), or an example inside a comparison. The comment above that function already says what should happen: a property of one part never becomes a length check, and the bullet goes to the judge.
The shipped intents usually name the part as the subject of the sentence: "The Summary is …", "OVERVIEW is at most 25 words", "L3 (one sentence, 60 words or fewer)". Nothing reads the subject, so the default applies. Conditions such as "Text over 300 characters was cut" read as requirements for the same reason.
That default suits an ask such as "write this in under 80 words". An Ideal State describes a deliverable with several named parts, and there most count phrases are about a part.
Two ways it could go
The choice is yours. We have not written a patch, and will send one for whichever you prefer.
- In
compileIntent, make a count phrase a code check only when its clause says it is about the whole output: "total", "overall", "whole", or a subject such as "the output" or "the response". Every other count bullet stays a judged question. It is a small change, and part limits are then judged instead of counted. - Let a bullet name the part it bounds, a heading or a frontmatter key, and measure only that part. That is more work, and it gives real code checks for section lengths.
- Ngôn ngữ chính
- TypeScript
- Star
- 19.9k
- Fork
- 2.5k
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Chuẩn bị môi trường
Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của danielmiessler/LifeOS
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 67/100
danielmiessler/LifeOS#2894 ·
Maintainer thường phản hồi trong vòng 5 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 82/100
danielmiessler/LifeOS#2891 ·
Maintainer thường phản hồi trong vòng 5 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100
danielmiessler/LifeOS#2888 ·
Maintainer thường phản hồi trong vòng 5 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
danielmiessler/LifeOS#2886 ·
Maintainer thường phản hồi trong vòng 5 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 68/100
danielmiessler/LifeOS#2848 ·
Maintainer thường phản hồi trong vòng 5 ngày
Tất cả issue của danielmiessler/LifeOS
Issue tương tự
-
[功能请求] 设置选项语言调整Đang mở功能请求
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 66/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 84/100
antithesishq/bombadil#473 ·
Maintainer thường phản hồi trong vòng 2 ngày
-
The structured output reference promises null results for refusals that carry validation detailsĐang mởbug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 74/100
mattermost/mattermost-mobile#10358 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
ready-for-agent
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 86/100
LucasSantana-Dev/Lucky#3257 ·
Maintainer thường phản hồi trong vòng 1 ngày