Upload a local run's report (simulator, emulator, device): the CLI side

Open
#442 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
flutter, go
Domain
cli, testing

Research direction

Start at the probe test --upload-report CLI path, then read hosted.CIFromEnv and hosted.Preflight, which the issue names as the current blockers. The requested work spans local Git and device facts, run sequencing, optional key enrollment, and updates to MCP descriptions, CLI and configuration docs, VS Code surfaces, and the changelog; the dirty-tree hash and keychain behavior are still open questions. Done means the acceptance criteria pass and all existing flags, report JSON, and envelope schema remain compatible.

Written by the indexing model from the issue text.

Description

area/cli area/cloud enhancement

What this is

Today probe test --upload-report only works inside GitHub Actions: hosted.CIFromEnv returns nothing elsewhere and hosted.Preflight refuses a payload without CI facts ("only --upload-dry-run works outside CI"). A developer who runs the suite on a local simulator, emulator or device has no way to send that run's report. This issue describes only what the CLI would need. No code yet; how a service treats such a run is out of scope here.

What the CLI would need

  1. A local mode. An explicit opt-in (probe test --upload-local or hosted.mode: local), never implied by --upload-report. The envelope says ci.provider: "local" (an additive value); everything stays optional and the report JSON and its schema value do not change. --upload-dry-run prints every field, as today.
  2. Git facts without CI variables. Commit (git rev-parse HEAD), branch (or HEAD when detached), and a dirty-tree flag plus a hash of the working tree so a run on uncommitted changes is never labelled as "the result of commit C". The hash definition must be stable across platforms and cheap on large repositories (open question: git stash create versus a hash of git diff HEAD plus untracked files).
  3. Device identity without personal identifiers. device_class and the platform/OS version already exist; add a short model string. Never a serial number or UDID, which identify a person's hardware.
  4. A run sequence. Outside CI there is no run id, so the client makes a monotonic one (for example milliseconds since the epoch plus a per-key counter) and a run_attempt of 1.
  5. Signer key enrolment (optional, later). An Ed25519 key generated on the developer's machine and stored in the OS keychain (macOS, Linux and Windows differ), enrolled once through a documented command (probe cloud enrol), with the envelope signed over its canonical bytes. The envelope carries only the key fingerprint (16 hex characters). No name or email goes into the envelope.
  6. Scrubbing and size limits are unchanged: relative paths, secret patterns, no artifacts, the 1 MiB / 5,000-result caps, retries on a busy service.

Constraints

  • Additive only: existing flags, config, the report JSON and the envelope schema keep working.
  • Nothing is sent unless the user opts in, and the docs state the hosted service is a preview and not live yet.
  • Any new flag or config key also updates the MCP descriptions, the CLI reference, the configuration page, the VS Code surfaces and the CHANGELOG.

Acceptance criteria (for a later implementation PR)

  • Tests: a local run builds an envelope with the git facts; a dirty tree sets the flag and the tree hash; a detached HEAD gives HEAD; a repository without commits and a non-git directory give a clear refusal, never a half-filled envelope.
  • No serial, UDID, user name, e-mail address or absolute path appears in the dry-run output.
  • Key custody: the private key never leaves the keychain, is never printed or logged, and signing fails closed when the keychain is unavailable.

Open questions

  • The dirty-tree hash definition (see 2).
  • Keychain access on each OS and a fallback for headless machines.
  • Whether the sequence needs a hash of the previous envelope so that a missing or reordered run is detectable on the client side.
Dominant language
Go
Stars
6
Forks
3
Avg merge
6h 28m
Merged PRs (30d)
362

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from AlphaWaveSystems/flutter-probe

All issues in AlphaWaveSystems/flutter-probe

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.