Slack notifications for inline alerts fail when the chart-explorer link passes Slack's 3000-character block limit
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- typescript
- Domain
- backend, observability
Research direction
Start with packages/api/src/tasks/checkAlerts/transports/slack.ts and inspect how the section block combines the link, title, and body; then read packages/api/src/tasks/checkAlerts/template.ts to trace how inline alert links are built. Reproduce the failure with a long inline alert link and check existing Slack transport tests. Done when oversized inline alert notifications are accepted by Slack while shorter alert links still work.
Written by the indexing model from the issue text.
Description
Summary
Slack notifications for inline alerts fail when the alert's chart config is large. The alert does fire, but HyperDX records WEBHOOK_ERROR: Failed to send notification to webhook "<name>" and nothing reaches Slack. The same alert set up as a tile alert posts fine.
Cause
For an inline alert, buildAlertMessageTemplateHdxLink (packages/api/src/tasks/checkAlerts/template.ts) links to the chart explorer with the alert's whole chartConfig URL-encoded in the query string (buildChartExplorerLink). Saved-search and tile alerts link to /search/<id> or the dashboard, which are short.
The Slack transport (packages/api/src/tasks/checkAlerts/transports/slack.ts) puts the link, the title and the body into a single section block:
blocks: [{ type: 'section', text: { type: 'mrkdwn', text: `*<${message.hdxLink} | ${message.title}>*\n${message.body}` } }]
Slack limits a section block's text to 3000 characters, and the URL counts toward that limit even though Slack only shows the title. So once the encoded config, the title and the body pass 3000 characters, Slack rejects the message.
Example (HyperDX 2.39.1)
An inline builder alert on a metrics source with 8 histogram series (quantile p99, one metricName each):
- Each series becomes about 280 characters of the link, because the stored config adds
aggCondition,aggConditionLanguage,isDeltaand similar fields and URL encoding roughly multiplies JSON by 1.4. - With series aliases, a descriptive name and display settings (
numberFormat,fitYAxisToData), the link was about 3100 characters. Every notification failed withWEBHOOK_ERROR. - After removing the aliases, display settings and most of the name, the link was about 2580 characters and the notification posted.
- A raw SQL inline alert with about 1700 characters of SQL produces a link of about 2500 characters and is right at the edge.
So a perfectly ordinary alert silently loses its notifications. The only sign is an executionErrors entry on the alert.
Suggestions
Any of these would fix it:
- Link inline alerts to something short, for example the alert itself (an alerts page or anchor by alert id), or a server-side stored chart that opens by id, instead of encoding the whole config in the URL.
- In the Slack transport, if the section text would pass 3000 characters, fall back to a shorter link (the alerts page) or put the link in a separate block or button. A button
urlallows up to 3000 characters on its own. - At minimum, warn when saving an inline alert whose notification link would exceed the Slack limit, and say in the
WEBHOOK_ERRORthat the message was too long.
Environment
- HyperDX 2.39.1 (ClickStack Helm chart 3.4.0), Slack incoming webhook
- Alert source: inline (created through the MCP
clickstack_save_alerttool / external API)
- Dominant language
- TypeScript
- Stars
- 10.1k
- Forks
- 527
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 186
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ClickHouse/ClickStack
-
external
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ClickHouse/ClickStack#3761 ·
Maintainers usually reply within 1 day
-
Metric chart with aggFn increase and a multi-column group-by fails: toString() gets several argumentsPossibly taken @CG-5228 claimed this 26 days ago. Openexternal
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
ClickHouse/ClickStack#3477 · 1 comment ·
Maintainers usually reply within 1 day
-
check-alerts worker: 13 error spans (mongodb.create, NamespaceExists) every minute because mongoose autoCreate is still onMay be free again A pull request for this issue was closed without being merged. Openbug external Needs Feedback
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ClickHouse/ClickStack#3308 · 1 comment ·
Maintainers usually reply within 1 day
-
count_distinct aggFn on metric tiles ignores valueExpression, always counts distinct metric ValuesPossibly taken @Harshul1484 claimed this 54 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ClickHouse/ClickStack#2311 ·
Maintainers usually reply within 1 day
-
enhancement Needs Feedback
Difficulty 5/5 Over a week Newbie friendliness 25/100
ClickHouse/ClickStack#3687 ·
Maintainers usually reply within 1 day
All issues in ClickHouse/ClickStack
Similar issues
-
Add: GTrekkerOpencheck:passed streams:add
Difficulty 1/5 Under an hour Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
area: ui Bug Payload OSS
Difficulty 1/5 Under an hour Newbie friendliness 88/100
payloadcms/payload#19839 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
fix(auth): add the email OTP verification step to Pollar loginPossibly taken @akansnoro claimed this today. OpenStellar Wave
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
safetrustcr/dApp-SafeTrust#584 · 10 comments ·
Maintainers usually reply within 1 day
-
The manager preset saves launch definitions without the required file and digest pairPossibly taken A pull request linked to this issue is open or already merged. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
Maintainers usually reply within 1 day