Add deterministic terminal transcript replay and stream tests

Open
  • Is something wrong?
#58 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
bash, typescript
Domain
frontend, testing

Research direction

Start by reading the existing line-tracker and suggestion-controller tests and helpers in the frontend logic to learn the replay interfaces. Create sanitised Bash, IOS-XE, NX-OS and TMOS transcript fixtures with synthetic secrets, then write deterministic replay tests using fake clocks and varied chunking. Done means all acceptance criteria pass: equivalent final state across chunk boundaries, no leaked secrets, and a reproducible command to add new transcripts.

Written by the indexing model from the issue text.

Description

enhancement help wanted p3 testing

Goal

Protect prompt detection and suggestions against realistic asynchronous terminal output without requiring live network devices.

Priority

6 of 6 in the testing improvement backlog. Raise priority if prompt/suggestion stream regressions become a frequent reported problem. Independent of native desktop automation for most cases.

Current gap

Existing line-tracker and suggestion-controller tests cover many specific cases. A reusable set of representative terminal transcripts replayed under varied chunking/timing would extend this protection across realistic device exchanges.

Scope

Create a small reviewed set of sanitised or synthetic Bash, IOS-XE, NX-OS and TMOS transcripts. Include prompt/mode changes, line editing, output bursts and interactive/secret exchanges. Run most scenarios against existing frontend logic with controlled clocks.

Acceptance criteria

  • Fixtures contain no real credentials, tokens, sensitive hostnames/addresses or unnecessary personal data; secret examples are synthetic.
  • Replay equivalent streams with different byte/chunk boundaries, including splits within escape sequences and multibyte text where supported.
  • Assert equivalent final reconstructed line, prompt/mode and suggestion state for equivalent output and input schedules.
  • Cover delayed partial prompts without premature detection and explicitly test timing-sensitive cases against the intended idle rules.
  • Cover large output bursts and verify suggestions remain appropriately suppressed.
  • Cover backspace, carriage-return redraws and reconciliation with observed remote echo.
  • Cover secret prompts and verify synthetic secret input does not enter suggestion state or captured logs, with recovery at the next normal prompt.
  • Cover interactive/full-screen-style output and verify required input passes through without spurious completions.
  • Use fake clocks and deterministic schedules rather than real sleeps or unseeded randomness.
  • Provide a reproducible command and a concise way to add a transcript when a new device regression is reported.

Boundaries

Reuse existing line-tracker/controller tests and helpers; do not introduce a second terminal parser or full device emulator. These logic tests complement, not replace, a few native terminal smoke checks. Chunk-boundary equivalence must not assume that intentional changes in idle timing produce identical intermediate states.

Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
6h 51m
Merged PRs (30d)
68

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from bonzo81/comnd

All issues in bonzo81/comnd

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.