Improve organic agent behavior when setting up Sentry

This is open design research, not a bounded change. Start with packages/cli/src/lib/detect-agent.ts and packages/cli/src/lib/agent-skills.ts to see how agents are detected and skills installed, then review the llms.txt agent-instructions block and the platform Markdown pages the issue names. Done means a written map of where agents get Sentry information, measured against the benchmark, with recommended steering surfaces checked against the no-growth-hacks guardrail.

When a developer asks a coding agent with no Sentry tooling installed to "set up Sentry", the agent mostly works Sentry out from first principles. It doesn't find or use the CLI, MCP, or skills. This issue covers the organic path: what agents actually do today, what pushes them toward the right setup (CLI-first, current SDK, the right features), and how we'll measure it.

Direction

• The remote-agent implementation of sentry init (e.g. "set up my GitHub repo with Sentry") is being deprioritized. It is the only hard requirement for a remote agent, as opposed to assuming people already have their own coding agent, and its ROI is low by comparison.
• This does not change the onboarding goal. The CLI is still the intended onboarding mechanism, and the CLI + MCP unification (#1357, getsentry/cli#1248) is the main bet.
• Open questions: do we bundle skills, do we need skills at all, is this a docs improvement, should docs be colocated (shipped with packages) or remote?
• Guardrail: help people adopt technology they want but don't yet understand. Don't push technology we want them to want (no growth hacks). Any steering toward CLI, MCP, or extra features has to hold up against this.

Observed organic behavior (one internal sample, 2026-10-14)

• Prompt: "set up Sentry for my web game" (browser JS), Opus 5.5, no plugin installed.
• Result: it worked, but only errors and tracing were set up. The CLI wasn't installed, MCP wasn't configured, and no plugin was installed.
• It picked an older SDK because v11 "looked sus". npm context at that time: 11.0.0 shipped 2026-09-23 and latest is 11.7.0 (2026-10-12). The v10 line was still publishing (10.76.2, 2026-10-08), and the next dist-tag still points at 11.0.0-rc.1. The agent's actual reasoning is unknown.
• Sources it used, per its own session summary: the npm registry (@sentry/browser versions and release dates) and the Sentry CDN (bundle names, integrity hashes). It did not read docs.sentry.io and ran no web searches.
• The @sentry/browser npm README (about 1.1 KB) contains only two docs links. It has no agent guidance and doesn't point to the CLI, MCP, or skills. One idea from the discussion: since agents look at the registry first, registry-side instructions could steer them.
• A second data point: asked how it would add Sentry, Claude described the ideal flow. It would inspect the project, fetch the official framework docs (preferring llms.txt or Markdown), use non-interactive npx @sentry/wizard flags or set things up by hand (interactive prompts often fail in agent shells), ask for the DSN, slugs, and token, and verify. The observed run skipped docs entirely, so what an agent says it will do is not a reliable proxy for what it does.
• The same answer said type definitions (.d.ts) are what it reads most in packages when writing config. JSDoc on each option (purpose, default, deprecation) could be high-leverage agent documentation that needs no new convention.

What we already have (verified 2026-10-14)

• Idea from the discussion: serve different content to agents on the docs site when we can detect them. Partly in place already:
  • docs.sentry.io returns Markdown for Accept: text/markdown and for .md URLs, and it also returns Markdown for a Claude-User UA. A bare axios/* UA with no Accept header gets HTML.
• docs.sentry.io/llms.txt has an "Instructions for AI Coding Assistants" block. It tells agents to install npx @sentry/agent-plugin install and use skills.sentry.dev/instrument and mcp.sentry.dev. It does not mention the sentry CLI, which conflicts with the CLI-first direction.
• The platform getting-started Markdown (e.g. /platforms/javascript.md) doesn't mention the CLI, MCP, or skills.
• The CLI already detects agents through env vars and the process tree (packages/cli/src/lib/detect-agent.ts, adapted from vercel/detect-agent). It uses this for cli setup, the init wizard, and telemetry, and it can install embedded skills (packages/cli/src/lib/agent-skills.ts).
• Benchmark: Value Discovery aims to swap harness, model, plugin, and repo and score tasks like "set up Sentry". It must work without the plugin, so the same prompt can be compared across models with or without any Sentry tooling. Related: #1383 (run init evals on CI).

Industry learnings

• Detecting agents on the web:
  • The Accept header is the most reliable signal so far. As of Feb 2026, Claude Code, Cursor, and OpenCode send text/markdown. Codex (ChatGPT-User UA), Gemini CLI (GoogleAgent-URLContext), Copilot, and Windsurf don't. User-Agent is inconsistent: Claude Code sends axios/*, and Cursor and OpenCode send a Chrome UA. (Checkly)
  • Cloudflare converts HTML to Markdown at the edge when the request asks for Markdown. It reports about 80% fewer tokens and adds x-markdown-tokens and content-signal headers. Vercel and Checkly report 99%+ reductions. (Cloudflare, Vercel)
  • Cryptographic identity is emerging: Web Bot Auth (HTTP message signatures) and Cloudflare's verified/signed-agent classification, which includes a "user-directed agent" category. (Cloudflare)
• Detecting agents locally: the vercel/detect-agent rules (env vars like CLAUDECODE, CURSOR_AGENT, AI_AGENT) are becoming the shared registry. Next.js uses detection in next dev to write managed AGENTS.md blocks.
• Always-on context vs. on-demand skills:
  • Vercel found an 8 KB docs index in AGENTS.md scored 100% on Next.js 16 evals. Skills scored 53%, the same as no docs, because they weren't invoked in 56% of runs. Explicit "use the skill" instructions raised skills to 79%, but results were sensitive to wording. (Vercel)
  • OpenAI likewise treats skill triggering as something to eval, not assume: the skill's name and description drive invocation. (OpenAI)
• Colocated, version-matched docs (Next.js 16.2+):
  • Next.js ships docs in node_modules/next/dist/docs/ and writes a managed AGENTS.md block ("This is NOT the Next.js you know… read the bundled docs") when it detects an agent.
  • It also offers per-page .md and llms.txt, agent-readable error pages linked from CLI output, next upgrade --agent, and opt-in agent feedback.
  • Skills are reserved for multi-step workflows, not knowledge lookups. (Next.js, evals)
• Agent-specific onboarding pages: Cloudflare docs link an "Agent setup" page from every page, with per-agent guides for Claude Code, Codex, Cursor, Copilot, and OpenCode covering skills and MCP. (Cloudflare)

Scope

• Map where agents actually get Sentry information in organic runs (package registries, READMEs, type definitions, CDN, docs, llms.txt, CLI output), using the benchmark rather than single samples or self-reports.
• Measure whether agents pick the current SDK, enable appropriate features, discover or install the CLI, and use CLI and MCP afterward.
• Evaluate the steering surfaces: package READMEs and registry metadata, JSDoc in type definitions, docs and llms.txt wording, CLI output when an agent is detected, colocated docs vs. remote docs vs. skills. Check each against the no-growth-hacks guardrail.
• Make agent-facing guidance consistent with the CLI-first direction (for example, the llms.txt plugin-first wording).

Unknowns

• Why the agent distrusted v11. Candidates are release cadence, the parallel v10 publishes, the stale next tag, or training-data priors.
• How representative one Opus sample is across other harnesses and models.
• Whether skills earn their place for setup at all, given the trigger-rate findings above.

via David Cramer.

--

View Junior Session [Sentry]