Session-grouped queries treat every sessionless event as one session (wrong results, ClickHouse OOM)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 52/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 技术栈
- clickhouse, sql, typescript
调研方向
Start with funnel.sql.ts, conversion.sql.ts, sankey.sql.ts, overview.sql.ts and pages.sql.ts, following the query names listed in the issue. Add the empty-session guard to per-session query paths and make sessionless view duration zero; leave profile grouping and the listed follow-ups out of scope. Done means sessionless events no longer join into shared paths or conversions, while profile-grouped results and view and visitor counts remain unaffected.
由索引模型根据 Issue 内容生成。
描述
Problem
Events tracked from a backend are stored with session_id = '' by design. This covers secret-authenticated /track calls with a server user agent when the profile has no live browser session (incoming-event-handler.ts falls back to the '' that getDeviceId returns), and events whose timestamp is in the past.
The queries that group, partition or join by session_id don't exclude ''. Every sessionless event in the project and date range therefore lands in one group, which the query treats as a single session:
| Surface | Query | Effect |
|---|---|---|
| Funnel, grouped by session (the default) | funnel.sql.ts funnelBase (GROUP BY session_id + windowFunnel) |
Step 1 from user A and step 2 from user B count as one completed funnel |
| Conversion, grouped by session | conversion.sql.ts conversionQuery |
Same as the funnel: conversions that never happened |
| Flows (sankey) | sankey.sql.ts (GROUP BY session_id + groupArray) |
One path stitched together from every server-side user's events. The start_event_sessions / end_event_sessions anchors match '' too |
| Overview → User Journey | overview.sql.ts topEntriesQuery / transitionsQuery (GROUP BY session_id + groupArray(path)) |
Fake entry pages and transitions, plus one array holding every sessionless screen view in the range |
| Page-filtered overview metrics, Pages table | overview.sql.ts metricsWithPageFilterQuery, pages.sql.ts topPagesQuery (lead() OVER (PARTITION BY session_id …)) |
A sessionless view's "duration" is the gap to the next sessionless view by anyone |
Profile-grouped funnels and conversions are not affected, since they group by profile_id.
OpenPanel already treats '' as "no session" elsewhere. The import pipeline only builds sessions from session_id != '' (and never for device = 'server'), and code migration 4-add-sessions filters session_id != ''. The analytics queries just never got the same guard.
Reproduction
-
Track two server-side events for two different profiles with a client secret and a non-browser user agent. Neither profile has a browser session:
for p in user-a:screen_view user-b:sign_up; do curl -sS -X POST "$API_URL/track" -H 'content-type: application/json' \ -H "openpanel-client-id: $CLIENT_ID" -H "openpanel-client-secret: $CLIENT_SECRET" \ -H 'user-agent: my-backend/1.0' \ -d "{\"type\":\"track\",\"payload\":{\"name\":\"${p#*:}\",\"profileId\":\"${p%%:*}\",\"properties\":{\"__path\":\"/x\"}}}" done -
SELECT profile_id, session_id, name FROM events WHERE project_id = '<project>' ORDER BY created_atshows both rows withsession_id = ''. -
Create a funnel
screen_view → sign_upgrouped by session. It shows 1 conversion, although no user did both steps. Grouped by profile it correctly shows 0. -
Open Flows starting at
screen_view. It drawsscreen_view → sign_up, a path nobody took. -
Send more sessionless
screen_views with different paths, then open Overview → User Journey. The paths are stitched together across users.
On a project that sends a lot of server-side screen_views, step 5 also shows the memory problem. All of those rows go into a single groupArray, which can't be split across threads or shards.
Impact
- Wrong numbers. Per-session funnels and conversions are inflated, and flows and user journeys show paths that don't exist. Nothing in the UI shows it, so the numbers look fine.
- Memory blow-up. The combined "session" grows with all sessionless traffic in the range. On 2026-10-20 a self-hosted ClickHouse was OOM-killed by the Overview User Journey query grouping about 97k sessionless
screen_viewrows into one session. A local reproduction with 1M sessionless rows: the user-journey grouping peaked at about 92 MB for one 1,000,000-element path. With the guard it peaked at about 9 MB, and the longest path was 5.
Proposed fix
Add session_id != '' wherever a query builds per-session state, and leave profile grouping alone:
- Funnel and conversion: add the filter to the scan only when the group is
session_id. - Sankey: add it to the path scan and to the start/end anchor CTEs.
- Overview user journey: add it to
ordered_events. - The
lead() OVER (PARTITION BY session_id)screen-view duration: give a sessionless view 0, the same as a session's last view, rather than the gap to someone else's event. View and visitor counts stay as they are.
Out of scope for this issue, possible follow-ups:
uniq(session_id)/countDistinct(session_id)session counts (the overview, realtime, widgets, the chart "sessions" segment) count''as one extra session. That's off by at most one and doesn't blow up memory.- The realtime map's "latest event per session" list can show one
''row. - The
session.performed_eventfilter'ssession_id IN (SELECT DISTINCT session_id …)matches every sessionless event once any sessionless event performed the step.
- 主要语言
- TypeScript
- 星标
- 7.2k
- 派生
- 523
- 平均合并
- 16 小时 15 分钟
- 30 天内合并 PR
- 30
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
Openpanel-dev/openpanel 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 76/100
Openpanel-dev/openpanel#579 ·
维护者通常 1 天内回复
-
ClickHouse retry logs print CLICKHOUSE_URL with its password可能已有人在做 @ppjsw 于 3 天前认领。 未关闭
难度 1/5 1-3 小时 新手友好度 92/100
Openpanel-dev/openpanel#551 · 1 条评论 ·
维护者通常 1 天内回复
-
login page needs refinement可能重新可做 关联的 PR 已关闭且未合并。 未关闭
难度 2/5 1-3 小时 新手友好度 68/100
Openpanel-dev/openpanel#495 ·
维护者通常 1 天内回复
-
难度 5/5 一周以上 新手友好度 35/100
Openpanel-dev/openpanel#563 ·
维护者通常 1 天内回复
-
Session replay: early chunks saved under the wrong session可能重新可做 关联的 PR 已关闭且未合并。 未关闭
难度 4/5 3-5 天 新手友好度 35/100
Openpanel-dev/openpanel#537 · 1 条评论 ·
维护者通常 1 天内回复
查看 Openpanel-dev/openpanel 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 66/100
db-ux-design-system/core-web#8613 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
bug:new
难度 1/5 1 小时以内 新手友好度 78/100
callstackincubator/simlock#934 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 68/100
gchq/LD-Explorer#286 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复