GitHub Action: observe runs are counted as surface=cli, because the Observe step never sets DIFFCI_SURFACE

Open
#284 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@FredApps is already working on this.

Since Nov 17, 2026.

  • #285 by @FredApps — open

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
github-actions, node.js, typescript

Research direction

Read the observe step in action.yml and compare its environment with action/verify.sh, which already exports DIFFCI_SURFACE=github-action. Check packages/diffci-cli/src/client/telemetry.ts for how the surface is read, then verify the Action metadata check in the issue passes. A linked open pull request (#285) already addresses this issue.

Written by the indexing model from the issue text.

Description

What went wrong?

The output or a command was wrong or confusing

DiffCI version

0.3.54 (the Action's default cli-version on main, 40aa308)

Command you ran

The public DiffCI/core change impact Action, default verify: "false". Its Observe step runs the published CLI:

npx --yes --ignore-scripts --no-audit --fund=false --package "@diffci.com/[email protected]" diffci observe --repo "$DIFFCI_REPO"

without DIFFCI_SURFACE=github-action in that step's environment (action.yml, the observe step).

Environment

Any GitHub Actions runner; Node 22 (the Action's default). Reproduced by reading action.yml and packages/diffci-cli/src/client/telemetry.ts on main (40aa308).

Output

surfaceOf in packages/diffci-cli/src/client/telemetry.ts only returns github-action when DIFFCI_SURFACE is that string. GITHUB_ACTIONS is used for the ci flag, not for surface. So an observation from the Action is sent to https://app.diffci.com/v1/usage-events with surface: "cli".

The opt-in fast check is already counted correctly: action/verify.sh exports DIFFCI_SURFACE=github-action. The GitLab component sets DIFFCI_SURFACE=gitlab-component, which telemetry counts. The Azure DevOps task sets DIFFCI_SURFACE=azure-devops-task, but telemetry's surfaceOf doesn't recognise that value and counts it as cli. That is a separate gap, filed as #303. Among the wrappers, only the Action's default path (observe) sets no surface at all.

A check that reads the Action metadata fails on main:

# from a worktree of main, after npm ci
node --input-type=module -e '
import { readFileSync } from "node:fs";
import { parse } from "yaml";
const action = parse(readFileSync("action.yml", "utf8"));
const observe = action.runs.steps.find((s) => s.id === "observe");
if (observe?.env?.DIFFCI_SURFACE !== "github-action") {
  console.error("observe env DIFFCI_SURFACE:", observe?.env?.DIFFCI_SURFACE);
  process.exit(1);
}
'
observe env DIFFCI_SURFACE: undefined
What you expected

Observe runs started by the GitHub Action are counted as surface: "github-action", the same as verify from that Action, and as the GitLab component already does for its observe path. (The Azure task's value isn't recognised yet; see #303.) Adoption by surface is then accurate. docs/telemetry.md already lists github-action as a surface the Action is supposed to report.

Dominant language
TypeScript
Stars
0
Forks
4
Avg merge
18h 41m
Merged PRs (30d)
48

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DiffCI/core

All issues in DiffCI/core

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.