LabelBatch questions show only the check id and a cut message, not the criterion or the passage the flag is about

Open
#2,763 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 5 days

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
56/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
typescript
Domain
ai

Research direction

Start in LIFEOS/TOOLS/LabelBatch.ts, especially pickFlags and the question line cited in the issue. Trace how batch items get their check data and whether the engine exposes a passage span; the issue suggests storing derived_from and probability, then showing a span or a marked best-match passage. Done means each question includes the criterion, relevant artifact passage, and probability.

Written by the indexing model from the issue text.

Description

What happens

A LabelBatch question shows only the artifact's basename, the check id and the first 200 characters of the finding message (LIFEOS/TOOLS/LabelBatch.ts, pickFlags: message: fd.message...slice(0, 200); the question line is `${i.n}. ${basename(i.artifact)}: flag "${i.check_id}" said: ${i.message}. Was this flag right? y/n`).

To answer y/n the principal has to open the artifact, find the sentence the flag is about, and, for a criterion judged against a source, find the passage of the source too. With ten questions per batch this is most of the work, and an answer given without that lookup is a weak label.

Expected

Each question carries what it takes to answer it: the criterion text, the passage of the artifact the flag is about, and the probability Glance gave.

Possible fix

Store the criterion text (derived_from) and the probability on the batch item, and show them in the question. When the engine can return the span it relied on, show that span; otherwise show the artifact passage around the best match of the criterion's key terms, marked as a guess.

🤖 Generated with Claude Code

Dominant language
TypeScript
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from danielmiessler/LifeOS

All issues in danielmiessler/LifeOS

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.