Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Run lexvm generate against the Anthropic API and document duration and cost

Open
  • Is something wrong?
#40 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, typescript
Domain
devtools, documentation, testing

Research direction

Start with packages/cli/src/generate/llm.ts, pipeline.ts, and fenced.ts, then read docs/authoring/pipeline.md and ADR 0007. Run the three specified generations against the Anthropic API and record duration, run.json token totals, repair rounds, cost, and outcomes. Add regression tests from recorded responses and handle output-limit stops automatically; done when a run passes every gate, measured results are documented, and the smoke workflow decision is recorded.

Written by the indexing model from the issue text.

Description

area: cli area: pipeline priority: high type: chore

Context

lexvm generate is tested only with a mocked client. "Time and cost" in docs/authoring/pipeline.md is qualitative. Long games may hit --max-tokens: a cut-off answer is only logged as a warning, and its unclosed last file block is dropped, which leaves the stage incomplete. Chat answers in the one-shot flow can be cut off the same way. The v0.3 generation workflow depends on a pipeline that has been run for real.

Scope

  • Three runs with a current model: German 15 minutes, English 30 minutes, German 45 minutes. Record duration, token usage (from run.json totals), repair rounds, cost at list prices and the outcome. Name the model only in the pull request, following the provenance policy.
  • Fix the defects found; add regression tests based on recorded responses (no secrets).
  • Handle max_tokens stops without manual work: request a continuation, or split large stages (for example the logic stage per file).
  • One-shot flow: tell users in the prompt and on the Create screen what to do when an answer is cut off.
  • Replace the qualitative cost text in docs/authoring/pipeline.md with a table of measured runs.
  • Decide on an optional manually triggered smoke workflow with an API key secret (a five-minute pack at low effort).
  • Generated packs go through normal review; they are not merged automatically.

Acceptance criteria

  • At least one run passes every gate; its pack, design/ and run.json are attached to a pull request.
  • docs/authoring/pipeline.md has a table of measured duration and cost.
  • An answer cut off at the output limit is completed automatically (test with a recorded response).
  • The smoke workflow is added or rejected with a reason recorded here.

References

  • packages/cli/src/generate/llm.ts
  • packages/cli/src/generate/pipeline.ts
  • packages/cli/src/generate/fenced.ts
  • docs/authoring/pipeline.md (Time and cost, Troubleshooting)
  • docs/adr/0007-llm-pipeline-byo-prompt-kit-and-api-generator.md
Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
28m
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from nullthrone/lexvm

All issues in nullthrone/lexvm

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.