research: how to understand Cortex adoption (installs, versions, retention) without being intrusive

Abierto
  • ¿Algo no está bien?
#1,321 2 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@mrsabath ya está trabajando en esto.

Desde el 7/10/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

laptop question
Feature Description

We cannot currently answer basic questions about Cortex adoption: how many people have installed
it, which versions are in use, how many installs are still running a week later. The only signal
today is anecdotal — someone mentions in chat that they installed it.

This issue is research, not implementation. The goal is to understand the available options
for obtaining adoption information, what each one costs in user trust, and what the honest
trade-offs are — and to produce a recommendation for discussion. No approach should be
implemented off the back of this issue. Whatever comes out of it gets its own issue, scoped
and agreed separately.

The reason for the care: Cortex sits in the request path and decrypts the user's agent traffic.
That earns a level of trust that is very easy to spend and very hard to get back. Users are
rightly sensitive about anything leaving their machine, and a tool positioned as "see what your
coding agent actually sends" undermines its own pitch if it quietly sends things itself. The
default assumption for this research is that intrusive collection is off the table, and the
burden is on any proposal to show it does not cross that line.

It is a legitimate outcome of this issue to conclude that we should collect nothing from
installed clients, and get what we need from other sources — or to accept that some questions
simply stay unanswered.

Proposed Solution

Investigate and write up the options. Not a plan to build any of them.

1. What do we actually need to know, and why?

  • List the specific questions worth answering (install count? version spread? retention?
    which agent integrations are used? platform mix?)
  • For each, state the decision it would inform. Questions that would not change any decision
    should be dropped rather than carried along
  • Separate "would genuinely change what we build" from "nice to know" — the second category
    does not justify any user-visible cost
  • Note what precision is actually required: an order of magnitude is often enough, and cheap
    approximations may beat exact counts that need instrumentation

2. Survey the approaches, including the ones that collect nothing

Zero-collection and off-machine sources first, since these carry no user cost:

  • Download and release-asset counts (GitHub release API, install-script fetches, container
    pulls, package-manager stats) — what they can and cannot tell us, and how badly CI and
    mirrors distort them
  • Repo-side signals: stars, forks, clones, issue and feedback volume, questions in chat
  • Asking people directly: surveys, a check-in with known users, opt-in registration for
    early adopters. Low-tech, honest, and possibly sufficient at our current scale
  • Update-check traffic, if an update check is something users would want anyway for its own
    sake — and whether using it for counting is a bait-and-switch

Then approaches that do involve the client, each assessed honestly rather than advocated for:

  • Fully opt-in reporting (explicit affirmative action, off by default, no dark patterns)
  • Local-only accounting the user can voluntarily share — Cortex keeps its own numbers, nothing
    leaves the machine unless someone chooses to send them
  • Aggregate or privacy-preserving techniques, and whether their complexity is remotely
    justified at our scale
  • Anonymous reporting — documented here specifically so its costs are on the record. Note
    that "anonymous" is a weaker claim than it sounds: IP addresses, timing and rare
    configuration combinations can re-identify small populations

3. Assess each against the trust bar

  • What leaves the machine, in concrete terms — not a category label
  • Whether it can be meaningfully described to a user in one sentence they would accept
  • Re-identification risk, especially at small user counts where uniqueness is high
  • Whether it is verifiable by a suspicious user, or requires taking our word for it
  • Reaction if it were discovered rather than announced — a useful test of whether we are
    comfortable with it
  • What it would do to Cortex's positioning as a transparency tool

4. Look at what comparable tools do — and how it went

  • How other local developer tools and proxies handle this (both the ones that collect and the
    ones that deliberately do not)
  • Cases where telemetry damaged a developer tool's standing, and what specifically triggered
    it — usually defaults, disclosure and discoverability rather than the payload itself
  • Any legal or policy constraints that apply to us

5. Deliverable

  • A written comparison of the options with the trade-offs stated plainly, including the
    do-nothing option as a real candidate
  • A recommendation, which may be "collect nothing from clients"
  • If any client-side approach is recommended: what it sends, the default, the disclosure, the
    off switch, and how a user verifies the claim — as a proposal for discussion, not a
    decision already taken
  • Open questions needing a product or maintainer call, flagged rather than resolved quietly
  • Follow-up issues filed for whatever is agreed. Nothing ships from this issue.
Additional Context

Part of the Cortex v0.9.0 release, Pillar 2 (Observability) of rossoctl/rossoctl#2514 — here
pointed at understanding adoption of the tool itself rather than at the agent's traffic.

Related:

  • cortex#898 — report statistics to a central collector. Overlapping concerns: its payload is
    traffic and cost rather than adoption, but it raises the same trust questions and should
    probably not be designed independently of whatever this research concludes
  • cortex#944 / cortex#945 — macOS and Linux install + service lifecycle (where install-time
    signals would originate, if any)
  • cortex#959 — first-run path, install to first useful reading (the drop-off we cannot currently
    see)
  • cortex#1236 / cortex#1237 — agentop configure surfaces (relevant to the integration-mix
    question)
  • cortex#1002 — Cortex being down can break the harness (any client-side network call would have
    to avoid this failure mode entirely)
Lenguaje dominante
Go
Estrellas
21
Forks
43
Merge medio
13 h 32 min
PR fusionados (30 d)
256

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de rossoctl/cortex

Todos los issues de rossoctl/cortex

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.