feat: add a product analytics plugin, with experimentation inside it (build on demand)
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 18/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Necesita aclaración
- Estado de actividad
- Activo
- Stack tecnológico
- shell
- Área
- developer-experience, tooling
Línea de trabajo
Do not implement yet. The issue is a gap record, not a spec: read #6306 (UX plugin boundary) and #6298 (UI/dashboards) first, then the open interview questions in this ticket. Work starts only after an owner interview confirms vendor tools do not cover the need. Done means a scoped plugin design agreed with the owner, not a first-pass patch.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem
The planned user-experience plugin (#6306) chooses what to measure about the experience, but nothing in this marketplace turns those choices into instrumentation, a tracking plan, funnels, cohorts, activation definitions or experiments. Today that work routes to whatever analytics tools a team already has installed. If a team needs vendor-neutral analytics work those tools do not cover, there is no home for it.
This issue records the gap so it is not forgotten. It is not a design. The decision taken during the UX plugin interview is to build this plugin only when repeated use shows a need the installed vendor tools do not meet, and the scope needs a detailed interview with the owner before any work.
Proposed scope
- Tracking plan and event taxonomy, built from the signals the UX plugin says must be captured.
- Activation and retention definitions, funnel and cohort questions, answered through the analytics tools the project already has.
- Online controlled experiments: design, guardrail metrics, sample ratio mismatch checks, and the peeking problem. Experimentation sits inside this plugin, not in a plugin of its own.
- Growth and business questions (activation, conversion) that go beyond user-centered measurement.
- Routing to installed vendor plugins and MCP servers rather than wrapping vendor APIs.
Out of scope (what UX keeps)
- Choosing experience metrics (HEART, Goals-Signals-Metrics).
- Attitudinal measures (standardized usability questionnaires) and task measures (success, time on task, errors).
- The list of HEART signals a tracking plan must capture. UX hands this list over; this plugin implements it.
- How dashboards look, which belongs to the UI plugin (#6298).
Evidence
Confidence labels follow the research behind the UX plugin. Vendor rows are what each vendor says about its own product, so they are single-publisher, MEDIUM at best, and dated 2026-10-04. Naming a vendor here records that a tool exists; it is not a recommendation.
- The boundary is drawn at the question, not the data. HEART turns large-scale behavioral data into user-centered metrics and adds surveys for attitudes; product analytics also answers growth questions; experiments add a causal layer with their own statistics. MEDIUM; the source covers only the HEART half (behavioral data plus surveys for attitudes); the growth and experiment halves and the placement of the line are judgment. Source: Rodden, Hutchinson and Fu, "Measuring the User Experience on a Large Scale" (CHI 2010), https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36299.pdf
- Guardrail metrics alert experimenters to violated assumptions, in a chapter on sample ratio mismatch and other trust-related guardrails. HIGH, single source. Source: Kohavi, Tang and Xu, "Trustworthy Online Controlled Experiments", https://www.cambridge.org/core/books/trustworthy-online-controlled-experiments/D97B26382EB0EB2DC2019A7A7B518F59/listing
- Checking fixed-horizon p-values repeatedly during a test invalidates them; sequential or always-valid methods address this. MEDIUM. Source: Johari et al., https://arxiv.org/abs/1512.04922
- UK central government services must publish four mandatory KPIs. HIGH, single source. That UX can cover the two user-facing ones (satisfaction, completion rate) and analytics the rest is judgment. Source: https://www.gov.uk/service-manual/measuring-success/data-you-must-publish
- No study found tests whether reaching an "activation" event causes retention; activation is a vendor and practitioner construct. MEDIUM (a gap: absence of evidence from one research pass).
- Vendor landscape, single-publisher, MEDIUM, as of 2026-10-04. Each of these vendors states it ships an official MCP server: Mixpanel (https://docs.mixpanel.com/docs/mcp), Amplitude (https://amplitude.com/docs/amplitude-ai/amplitude-mcp), PostHog (https://posthog.com/docs/model-context-protocol), Google Analytics 4, labeled Experimental in its README (https://github.com/googleanalytics/google-analytics-mcp) and read-only per Google's docs (https://developers.google.com/analytics/devguides/MCP), Statsig (https://docs.statsig.com/integrations/mcp/overview), Optimizely (https://docs.optimizely.com/experimentation-strategy/docs/install-optimizely-experimentation-mcp-server), Contentsquare (https://contentsquare.com/blog/introducing-model-context-protocol-mcp/), which states Hotjar is now part of Contentsquare (https://contentsquare.com/hotjar/free-mcp/), and FullStory, labeled beta under Early Access (https://developer.fullstory.com/mcp/faq/). Some also publish Claude Code plugins or skills on top of the server, per the vendors.
- Plan and admin gates, as stated by the vendors: Mixpanel's docs say an org admin must enable MCP except on new Free and Growth accounts created after 2026-08-01; FullStory says its server needs StoryAI and an admin toggle; Amplitude gates read and write by role-based permissions; Optimizely requires Opal. Single-publisher, MEDIUM, and each must be rechecked against the vendor's changelog before use.
- Anthropic's knowledge-work design plugin lists product analytics as a connector placeholder with no bundled server, and its skills fall back to pasted input. Observed in the plugin source. Source: https://github.com/anthropics/knowledge-work-plugins/blob/main/design/CONNECTORS.md
Open questions for a human interview
- Which analytics and experimentation tools do the teams that would use this run, and are their MCP servers enabled or admin-gated for them?
- What need would the vendor plugins not meet? Vendor-neutral tracking-plan authoring across several tools is one candidate. What signal would show the need is real?
- Who owns the tracking plan: product, engineering, analytics, or design?
- Does experiment design ship in the first version, or only analysis of experiments run elsewhere?
- What privacy review does event data need, and who signs it off?
- Should the plugin own instrumentation for apps that have no analytics vendor at all?
- How do results flow back to the UX plugin (for example, a funnel drop-off that needs a usability study)?
Related
- #6306 (user-experience plugin; this is the analytics sibling its boundary routes to)
- #6298 (UI plugin; owns dashboard presentation)
🤖 Generated with Claude Code
- Lenguaje dominante
- Shell
- Estrellas
- 22
- Forks
- 2
- Merge medio
- 5 h 4 min
- PR fusionados (30 d)
- 825
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de melodic-software/claude-code-plugins
-
good first issue needs-triage priority: medium
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
melodic-software/claude-code-plugins#6499 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
needs-human needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
melodic-software/claude-code-plugins#6496 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
good first issue needs-triage priority: low
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
melodic-software/claude-code-plugins#6390 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
good first issue needs-triage priority: low
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
melodic-software/claude-code-plugins#6386 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
needs-human needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
melodic-software/claude-code-plugins#6363 · 1 comentario ·
Los mantenedores suelen responder en 1 día
Todos los issues de melodic-software/claude-code-plugins
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
collabnix/awesome-mcp-lists#179 ·
Los mantenedores suelen responder en 1 día
-
electron tech debt
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
alunduil/alunduil-chezmoi#870 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 1 día
-
package-update
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
oSoWoSo/vOid_Community_repOsitory#268 · 1 comentario ·
Los mantenedores suelen responder en 1 día