analyze_issue_with_seer can't replace a completed Seer run

Open
  • Is something wrong?
#2,273 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
typescript
Domain
tooling

Research direction

Read packages/mcp-core/src/tools/catalog/analyze-issue-with-seer.ts around L149 (the startAutofix gate), L209-225 (the path that returns an existing terminal run), and the input schema at L83-92 with its Results are cached description at L80. Check the related getsentry/toolkit#1270 runId guard before changing the branch, since the new parameter has to coexist with it and with the fact that starting a run spends Seer credits. Done means a caller can explicitly request a fresh analysis, instruction is honoured or rejected rather than silently dropped, and the tool's existing tests are updated to cover both the reuse and refresh paths.

Written by the indexing model from the issue text.

Description

MCP mcp-feedback-pattern

Pattern: once an issue has a completed Seer run, analyze_issue_with_seer always returns that run. Callers can't start a fresh analysis through MCP, even when the stored root cause is stale (for example, it points at code that has since been fixed and shipped), and the instruction argument is dropped without notice.

Reports: 1 (user feedback, MCP server 0.42.0, analyze_issue_with_seer, impact: blocked)

Current behavior (packages/mcp-core/src/tools/catalog/analyze-issue-with-seer.ts @ 516cad4)

  • L149: the tool calls startAutofix only when there's no previous run or the previous run's status is error.
  • L209-225: any other terminal run (completed, waiting for input) is returned as-is with Found existing analysis (Run ID: …). params.instruction is only passed to startAutofix (L155), so on this path it is silently ignored.
  • The input schema (L83-92) has no way to ask for a new run, and the description says Results are cached - subsequent calls return instantly (L80).

Sentry API (getsentry/sentry@fe43ef0)

  • POST /issues/{id}/autofix/ starts a new run whenever neither run_id nor sentry_run_id is passed (src/sentry/seer/endpoints/group_ai_autofix.py L286-299). An existing completed run doesn't block it.
  • GET /issues/{id}/autofix/ returns the group's latest run (get_autofix_agent_state → fetch_latest_run_state, src/sentry/seer/autofix/autofix_agent.py L558-575). So the gap is on the MCP side only.

Evidence from the report

  • The sampled analyze_issue_with_seer call made GET …/issues/{id}/ and GET …/issues/{id}/autofix/ and no POST. It returned Found existing analysis (Run ID: …) / ## Analysis Complete with the old root cause. That sampled call had no instruction argument; the reporter says an earlier call did pass one. We sample 30% of traces, so that call may not have been recorded.

Reporter's proposal: a force/refresh parameter on analyze_issue_with_seer.

Related: getsentry/toolkit#1270 adds a runId guard on the same code path that stops the tool from starting a new run. Any change here needs to fit with it. Starting a new run spends Seer credits, which is likely why the tool reuses the existing run today.

via automation evt_905b9c478746fd9deeb853f85ad7441b · Run ID dispatch_603ac936cce8aaa570e9b7eae0aa9519.

--

View Session [Sentry]

Dominant language
TypeScript
Stars
961
Forks
179
Avg merge
15h 33m
Merged PRs (30d)
476

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from getsentry/toolkit

All issues in getsentry/toolkit

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.