[Bug]: SpannerIO Change Streams may skip ChildPartitionsRecord when advancing tracker to artificial query end timestamp
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 45/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- google-cloud, java
调研方向
The issue is in QueryChangeStreamAction.java around line 360. Start by understanding the restriction tracker and how it claims timestamps. Look at the logic for unbounded queries when stopAfterQuerySucceeds is false. The fix is to avoid advancing the tracker to the artificial changeStreamQueryEndTimestamp and instead leave it at the last claimed position. Run tests related to Spanner Change Streams to verify the fix doesn't break existing behavior.
由索引模型根据 Issue 内容生成。
描述
What happened?
When reading from an unbounded Spanner Change Stream (or a Mutable Change Stream with a capped query end timestamp), QueryChangeStreamAction issues change stream queries with an artificial changeStreamQueryEndTimestamp (now + 2 minutes).
Previously, when the query completed and needed to resume (!stopAfterQuerySucceeds), QueryChangeStreamAction called tracker.tryClaim(changeStreamQueryEndTimestamp) before returning ProcessContinuation.resume():
In a very rare timing corner case where a change stream query finishes without returning the ChildPartitionsRecord at the end of a partition's range, claiming changeStreamQueryEndTimestamp can advance the restriction tracker past the partition's actual end timestamp. On the next continuation, the query resumes from changeStreamQueryEndTimestamp + 1ns, which falls outside the partition's valid timestamp range and returns an out-of-range start_timestamp error, causing QueryChangeStreamAction to mark the partition FINISHED before scheduling its child partitions.
Workaround & Scope
This is a client-side workaround in Apache Beam for this rare edge case while a fix is being implemented on the Spanner server side:
- Unbounded queries: When
!stopAfterQuerySucceeds, leaving the restriction tracker at the last claimed position (from the last processed data or heartbeat record) instead of advancing tochangeStreamQueryEndTimestampensures the subsequent query resumes fromlastClaimedTimestamp + 1nsand reads any remaining records (includingChildPartitionsRecord) before the partition ends. This workaround addresses the issue for unbounded queries. - Bounded queries: For bounded queries where
changeStreamQueryEndTimestampreachesendTimestamp(stopAfterQuerySucceeds == true), this client-side workaround does not apply and will be addressed by the Spanner server-side fix.
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
- Component: Java SDK
- Component: IO Connectors
- 主要语言
- Java
- 星标
- 8.7k
- 派生
- 4.7k
- 平均合并
- 2 天 8 小时
- 30 天内合并 PR
- 246
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
apache/beam 的其他 Issue
-
[Bug]: Row.toString throws for an ITERABLE field that is not backed by a List可能已有人在做 @PDGGK 于 56 天前认领。 未关闭java P3
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
apache/beam#39624 · 2 个 reaction ·
维护者通常 1 天内回复
-
[Failing Test]: JmsIOTest. testCheckpointMark flaky可能已有人在做 @mxtymoshyk 于 13 天前认领。 未关闭bug failing test flake P2 pinned tests
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
[Bug]: PubsubIO used in batch incorrect batch cutoff size可能已有人在做 @1fanwang 于 44 天前认领。 未关闭bug io P3 pinned pubsub
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
build P3 sub-task
难度 1/5 1 小时以内 新手友好度 68/100
维护者通常 1 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
Netcracker/qubership-integration-platform#1046 ·
维护者通常 2 天内回复
-
`check_java_version()` fails when Java path contains spaces (Windows / Git Bash, `C:\Program Files`)未关闭
难度 2/5 1-3 小时 新手友好度 68/100
-
Fix Math.ceilDiv wrong result for exact positive divisions可能已有人在做 @pamod-madubashana 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
scala-native/scala-native#5094 ·
维护者通常 1 天内回复
-
[Bug] AI unread message badge counts a batch of new bubbles as one message可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 74/100
apache/rocketmq-dashboard#5784 ·
维护者通常 3 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
PCL-Community/PCL-CE#3658 ·
维护者通常 1 天内回复