feat: add a product analytics plugin, with experimentation inside it (build on demand)
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 18/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Da chiarire
- Stato di attività
- Attiva
- Stack tecnologico
- shell
- Ambito
- developer-experience, tooling
Direzione di ricerca
Do not implement yet. The issue is a gap record, not a spec: read #6306 (UX plugin boundary) and #6298 (UI/dashboards) first, then the open interview questions in this ticket. Work starts only after an owner interview confirms vendor tools do not cover the need. Done means a scoped plugin design agreed with the owner, not a first-pass patch.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem
The planned user-experience plugin (#6306) chooses what to measure about the experience, but nothing in this marketplace turns those choices into instrumentation, a tracking plan, funnels, cohorts, activation definitions or experiments. Today that work routes to whatever analytics tools a team already has installed. If a team needs vendor-neutral analytics work those tools do not cover, there is no home for it.
This issue records the gap so it is not forgotten. It is not a design. The decision taken during the UX plugin interview is to build this plugin only when repeated use shows a need the installed vendor tools do not meet, and the scope needs a detailed interview with the owner before any work.
Proposed scope
- Tracking plan and event taxonomy, built from the signals the UX plugin says must be captured.
- Activation and retention definitions, funnel and cohort questions, answered through the analytics tools the project already has.
- Online controlled experiments: design, guardrail metrics, sample ratio mismatch checks, and the peeking problem. Experimentation sits inside this plugin, not in a plugin of its own.
- Growth and business questions (activation, conversion) that go beyond user-centered measurement.
- Routing to installed vendor plugins and MCP servers rather than wrapping vendor APIs.
Out of scope (what UX keeps)
- Choosing experience metrics (HEART, Goals-Signals-Metrics).
- Attitudinal measures (standardized usability questionnaires) and task measures (success, time on task, errors).
- The list of HEART signals a tracking plan must capture. UX hands this list over; this plugin implements it.
- How dashboards look, which belongs to the UI plugin (#6298).
Evidence
Confidence labels follow the research behind the UX plugin. Vendor rows are what each vendor says about its own product, so they are single-publisher, MEDIUM at best, and dated 2026-10-04. Naming a vendor here records that a tool exists; it is not a recommendation.
- The boundary is drawn at the question, not the data. HEART turns large-scale behavioral data into user-centered metrics and adds surveys for attitudes; product analytics also answers growth questions; experiments add a causal layer with their own statistics. MEDIUM; the source covers only the HEART half (behavioral data plus surveys for attitudes); the growth and experiment halves and the placement of the line are judgment. Source: Rodden, Hutchinson and Fu, "Measuring the User Experience on a Large Scale" (CHI 2010), https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36299.pdf
- Guardrail metrics alert experimenters to violated assumptions, in a chapter on sample ratio mismatch and other trust-related guardrails. HIGH, single source. Source: Kohavi, Tang and Xu, "Trustworthy Online Controlled Experiments", https://www.cambridge.org/core/books/trustworthy-online-controlled-experiments/D97B26382EB0EB2DC2019A7A7B518F59/listing
- Checking fixed-horizon p-values repeatedly during a test invalidates them; sequential or always-valid methods address this. MEDIUM. Source: Johari et al., https://arxiv.org/abs/1512.04922
- UK central government services must publish four mandatory KPIs. HIGH, single source. That UX can cover the two user-facing ones (satisfaction, completion rate) and analytics the rest is judgment. Source: https://www.gov.uk/service-manual/measuring-success/data-you-must-publish
- No study found tests whether reaching an "activation" event causes retention; activation is a vendor and practitioner construct. MEDIUM (a gap: absence of evidence from one research pass).
- Vendor landscape, single-publisher, MEDIUM, as of 2026-10-04. Each of these vendors states it ships an official MCP server: Mixpanel (https://docs.mixpanel.com/docs/mcp), Amplitude (https://amplitude.com/docs/amplitude-ai/amplitude-mcp), PostHog (https://posthog.com/docs/model-context-protocol), Google Analytics 4, labeled Experimental in its README (https://github.com/googleanalytics/google-analytics-mcp) and read-only per Google's docs (https://developers.google.com/analytics/devguides/MCP), Statsig (https://docs.statsig.com/integrations/mcp/overview), Optimizely (https://docs.optimizely.com/experimentation-strategy/docs/install-optimizely-experimentation-mcp-server), Contentsquare (https://contentsquare.com/blog/introducing-model-context-protocol-mcp/), which states Hotjar is now part of Contentsquare (https://contentsquare.com/hotjar/free-mcp/), and FullStory, labeled beta under Early Access (https://developer.fullstory.com/mcp/faq/). Some also publish Claude Code plugins or skills on top of the server, per the vendors.
- Plan and admin gates, as stated by the vendors: Mixpanel's docs say an org admin must enable MCP except on new Free and Growth accounts created after 2026-08-01; FullStory says its server needs StoryAI and an admin toggle; Amplitude gates read and write by role-based permissions; Optimizely requires Opal. Single-publisher, MEDIUM, and each must be rechecked against the vendor's changelog before use.
- Anthropic's knowledge-work design plugin lists product analytics as a connector placeholder with no bundled server, and its skills fall back to pasted input. Observed in the plugin source. Source: https://github.com/anthropics/knowledge-work-plugins/blob/main/design/CONNECTORS.md
Open questions for a human interview
- Which analytics and experimentation tools do the teams that would use this run, and are their MCP servers enabled or admin-gated for them?
- What need would the vendor plugins not meet? Vendor-neutral tracking-plan authoring across several tools is one candidate. What signal would show the need is real?
- Who owns the tracking plan: product, engineering, analytics, or design?
- Does experiment design ship in the first version, or only analysis of experiments run elsewhere?
- What privacy review does event data need, and who signs it off?
- Should the plugin own instrumentation for apps that have no analytics vendor at all?
- How do results flow back to the UX plugin (for example, a funnel drop-off that needs a usability study)?
Related
- #6306 (user-experience plugin; this is the analytics sibling its boundary routes to)
- #6298 (UI plugin; owns dashboard presentation)
🤖 Generated with Claude Code
- Lingua principale
- Shell
- Stelle
- 22
- Fork
- 2
- Merge medio
- 5h 11m
- PR unite (30g)
- 838
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di melodic-software/claude-code-plugins
-
good first issue needs-triage priority: medium
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
melodic-software/claude-code-plugins#6631 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
needs-triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
melodic-software/claude-code-plugins#6547 ·
I maintainer di solito rispondono entro 1 giorno
-
needs-triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
melodic-software/claude-code-plugins#6535 ·
I maintainer di solito rispondono entro 1 giorno
-
test_comment_census.py: SccArgv flag-shaped-filename test errors on Windows (#!/bin/sh scc shim)Apertagood first issue needs-triage priority: low
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
melodic-software/claude-code-plugins#6532 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
good first issue needs-triage priority: low
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
melodic-software/claude-code-plugins#6390 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di melodic-software/claude-code-plugins
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
gnosis/gnosis_vpn#540 ·
I maintainer di solito rispondono entro 1 giorno
-
bug milestone-qa
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
lognorman20/monaco#3901 ·
-
Lid close does not lock the session on Apple Silicon (lid-close bind skips omarchy-system-lid-close)Aperta
Difficoltà 1/5 1-3 ore Idoneità per principianti 90/100
omacom/omarchy-mac#701 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
A 20.x release after 21.0.0 would move `latest` back to 20.x, and `next` stays on the release candidateForse già presa @armando-navarro l’ha presa oggi. Apertacomp: build/pipeline type: bug version: current (v17+)
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
angular/angularfire#3790 ·
I maintainer di solito rispondono entro 3 giorni