Copilot desktop app activity stopped being attributed to `copilot_app` in usage metrics

Open
#4,024 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github
Domain
analytics

Research direction

No source file or test is identified in the issue. Start by reproducing the organization users-1-day report for a desktop app user and compare the app and CLI attribution fields; the issue is done when desktop activity is attributed to copilot_app, retaining CLI attribution only when both surfaces are involved.

Written by the indexing model from the issue text.

Description

triage
Short summary

After a Copilot desktop app update, organization usage reports stopped attributing activity from some desktop app users to the copilot_app surface.

Affected version or release

1.1.21

Installation context

All people in the org using Github Copilot app.

What happened?

After a Copilot desktop app update, organization usage reports stopped attributing activity from some desktop app users to the copilot_app surface.

Before the update, desktop app sessions produced all expected signals:

  • used_copilot_app: true
  • totals_by_copilot_app
  • A copilot_app entry in totals_by_feature
  • In some cases, simultaneous CLI attribution because the app uses the CLI runtime internally

After the update, the same users’ activity continued to be reported, but only as CLI activity:

  • used_cli: true
  • totals_by_cli present
  • copilot_cli feature activity present
  • used_copilot_app: false
  • totals_by_copilot_app absent
  • No copilot_app feature entry

❗ Comparing weekdays before and after the transition:

  • Reported Copilot App active users fell 78%
  • Copilot App requests fell 94%
  • Copilot App generations fell 96%
  • CLI active users increased 42%
  • CLI requests increased 97%
  • CLI prompts increased 215%
  • CLI-only attribution increased 196%
  • Simultaneous app-and-CLI attribution fell 89%

The organization-level report reconciles exactly with the per-user rows, so this is not a downstream dashboard or aggregation problem. The incorrect attribution originates in GitHub’s telemetry or reporting pipeline.

Steps to reproduce
  1. Use the GitHub Copilot desktop app through an organization-managed Copilot license.
  2. Download the organization users-1-day usage report.
  3. Locate the user’s activity for that day.
  4. Observe that desktop app activity is reported under used_cli and totals_by_cli.
  5. Observe that used_copilot_app is false, while totals_by_copilot_app and the copilot_app feature row are absent.
Expected behavior

Activity initiated in the GitHub Copilot desktop app must be attributed to the copilot_app surface, even if the app uses the Copilot CLI runtime internally.

If both surfaces are intentionally involved, the report should retain both signals, as it did previously and as GitHub’s documented example schema supports.

Additional context

‼️ This is not acceptable for a paid organization product. We rely on these metrics to measure Copilot App adoption and justify substantial licensing and AI-credit expenditure. Reporting desktop app usage as CLI-only materially misrepresents product adoption and makes the organization metrics unreliable.

Dominant language
No language data
Stars
2.1k
Forks
157
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github/app

All issues in github/app

Similar issues

More Data Visualization issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.