alert metrics CLI doesn't support detector/workflow log alerts and list/view are inconsistent

Open
#1,142 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
typescript

Research direction

Start at the sentry alert metrics create, list, and view entry points and compare their legacy /alert-rules/ requests with the detector and workflow endpoints described in the issue. Reproduce the logs/EAP create and list/view cases, then define the supported model and verify that help text, creation, listing, and viewing are consistent.

Written by the indexing model from the issue text.

Description

bug enhancement

Summary

sentry alert metrics currently leads users/agents into legacy metric alert-rule behavior and does not support the new detector/workflow model required for log/EAP alerts.

In a Next.js project (demo/ai-engineer-conf) I tried to create log-based operational alerts for structured logs (trace_item_log). The CLI exposed sentry alert metrics create/list/view, but the behavior was confusing and partially broken for this use case.

Repro / observed behavior

1. create rejects logs as a dataset
sentry alert metrics create demo \
  --name 'AI chat failures' \
  --project ai-engineer-conf \
  --dataset logs \
  --query 'action:ai.chat (result:error OR result:stream_error)' \
  --aggregate 'count()' \
  --time-window 5 \
  --trigger '{"alertThreshold":1,"resolveThreshold":0,"label":"critical","actions":[{"id":"sentry.mail.actions.NotifyEmailAction","targetType":"Team","targetIdentifier":"2279506"}]}'

Returns:

Error: dataset must be one of: errors, transactions, sessions, events, spans, metrics.

However, log alerts are supported via the new monitors/alerts API using:

{
  "dataset": "events_analytics_platform",
  "eventTypes": ["trace_item_log"]
}
2. create trigger shape differs from API shape

The CLI help examples use legacy action ids like:

{"id":"sentry.mail.actions.NotifyEmailAction","targetType":"Team","targetIdentifier":1}

When posting directly to /api/0/organizations/{org}/alert-rules/, the API expects metric alert trigger actions shaped like:

{"type":"email","targetType":"team","targetIdentifier":"2279506"}
3. list does not list log/EAP alert rules that view can view

After creating a legacy EAP/log alert rule via direct API:

sentry alert metrics view demo/441020 --json

worked and returned the alert rule, including:

{
  "id": "441020",
  "dataset": "events_analytics_platform",
  "eventTypes": ["trace_item_log"]
}

But:

sentry alert metrics list demo/ --json --fresh

returned:

{"data":[],"hasMore":false,"hasPrev":false}

Direct API listing did return it:

sentry api '/api/0/organizations/demo/alert-rules/?project=4511626979573760'
4. The command appears to use legacy /alert-rules/, but we should be using detector/workflow

For the actual fix, I had to use the new model directly:

  • POST /api/0/organizations/demo/projects/ai-engineer-conf/detectors/
  • POST /api/0/organizations/demo/workflows/

Detector data source example that worked:

{
  "name": "AI chat failures",
  "type": "metric_issue",
  "dataSources": [
    {
      "aggregate": "count()",
      "dataset": "events_analytics_platform",
      "environment": null,
      "eventTypes": ["trace_item_log"],
      "query": "action:ai.chat (result:error OR result:stream_error)",
      "queryType": 1,
      "timeWindow": 300,
      "extrapolationMode": "unknown"
    }
  ],
  "config": {"detectionType": "static"},
  "conditionGroup": {
    "logicType": "any",
    "conditions": [
      {"type": "gt", "comparison": 0, "conditionResult": 75},
      {"type": "lte", "comparison": 0, "conditionResult": 0}
    ],
    "actions": []
  }
}

Workflow connection example that worked:

{
  "name": "AI Engineer operational alerts",
  "detectorIds": [7659187],
  "triggers": {
    "logicType": "any-short",
    "conditions": [
      {"type": "first_seen_event", "comparison": true, "conditionResult": true},
      {"type": "reappeared_event", "comparison": true, "conditionResult": true},
      {"type": "regression_event", "comparison": true, "conditionResult": true}
    ]
  },
  "actionFilters": [
    {
      "logicType": "all",
      "conditions": [
        {"type": "issue_priority_greater_or_equal", "comparison": 75, "conditionResult": true}
      ],
      "actions": [
        {
          "type": "email",
          "integrationId": null,
          "data": {},
          "config": {
            "targetType": "team",
            "targetDisplay": null,
            "targetIdentifier": "2279506"
          },
          "status": "active"
        }
      ]
    }
  ]
}

Expected behavior

Either:

  1. sentry alert metrics should support the new detector/workflow-backed alert model for log/EAP alerts, including logs or events_analytics_platform + trace_item_log, and list/view should be consistent.

or

  1. The command should clearly state that it manages legacy alert rules only, and the CLI should expose separate commands for new monitors/detectors/workflows.

Environment

sentry --version
# 0.39.0-dev.1782404180
Dominant language
TypeScript
Stars
121
Forks
14
Avg merge
22h 3m
Merged PRs (30d)
94

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from getsentry/cli

All issues in getsentry/cli

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.