fix(api): make WatchSandbox loss-aware and resumable
@letv1nnn 已经在做这个了。
开始于 2026年9月15日。
评估
这个 Issue 还没有评估数据。
描述
User Story
As an API or SDK client monitoring a sandbox, I want watch streams to report recoverable loss and resume from a known cursor, so that transient lag or reconnects do not silently lose state or force me to restart observation from scratch.
Problem Statement
SandboxStreamEvent includes a SandboxStreamWarning payload intended for missed-message warnings. The current broadcast receive path converts lag into a RESOURCE_EXHAUSTED stream error and terminates. The request also has no resume cursor, so a reconnect cannot request events after the last successfully processed item.
The separate stop_on_terminal implementation now recognizes ERROR; that corrected behavior needs regression coverage but is not the remaining defect.
Impact / Why This Matters
Slow consumers and transiently disconnected SDK clients lose observability precisely when event volume is high. Polling GetSandbox can recover only the latest sandbox snapshot, not missed log or platform events. Callers cannot distinguish a complete replay from a gap or suppress duplicates reliably after reconnecting.
Proposed Design
Give watch events an ordered resumable position and allow requests to resume after a previously observed cursor. When the server can continue after broadcast lag, emit SandboxStreamWarning with the lost range and continue from the next available item. When history is no longer recoverable, terminate with a documented status and enough structured information for the client to restart safely.
Define cursor scope, retention, ordering across status/log/platform event sources, duplicate behavior, and terminal-phase completion. SDK helpers should reconnect and resume without hiding unrecoverable gaps.
Acceptance Criteria
- Recoverable broadcast lag emits a warning event rather than immediately terminating the stream.
- Every resumable event exposes an ordered cursor or equivalent continuation position.
- A client can reconnect and request events after its last processed cursor.
- Recoverable and unrecoverable gaps have distinct documented behavior.
- Ordering across sandbox snapshots, logs, and platform events is specified.
- SDK helpers expose warnings and support safe resume without silently dropping or duplicating events.
- Tests cover lag, reconnect, cursor expiry, duplicate suppression, terminal READY/COMPLETED/STOPPED/ERROR behavior, and client cancellation.
Reproduction Steps
- Start
WatchSandboxwith one or more live event sources. - Allow the server-side broadcast receiver to lag beyond channel capacity.
- Observe that the gateway converts
RecvError::LaggedtoRESOURCE_EXHAUSTEDand returns, rather than emitting the declared warning payload.
Environment
- OpenShell: source investigation on
mainat69a05ebb3 - OS: not applicable
- Runtime, deployment, or integration: gateway
WatchSandboxbroadcast streams
Alternatives Considered
Terminate on every lag and require clients to poll current state. This cannot recover logs or events and does not use the existing warning contract. Emit warnings without adding cursors. This makes loss visible but still gives reconnecting clients no reliable continuation point.
Agent Investigation
The lag conversion lives in crates/openshell-server/src/sandbox_watch.rs, and the watch producer returns after sending the resulting status. SandboxStreamWarning remains in proto/openshell.proto.
Related: #2565. Source audit: https://gist.github.com/mrunalp/e80942c1544a0225ee588796a41ab30b.
- 主要语言
- Rust
- 星标
- 8.7k
- 派生
- 1.3k
- 平均合并
- 2 天 6 小时
- 30 天内合并 PR
- 297
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
NVIDIA/OpenShell 的其他 Issue
-
area:docs
难度 1/5 1 小时以内 新手友好度 88/100
-
state:triage-needed
难度 2/5 1-3 小时 新手友好度 82/100
-
area:cli state:validated
难度 2/5 1-3 小时 新手友好度 72/100
-
state:triage-needed
难度 1/5 1 小时以内 新手友好度 90/100
-
area:build spike state:review-ready state:stale
难度 2/5 半天 新手友好度 68/100
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 75/100
-
state:needs triage
难度 2/5 1-3 小时 新手友好度 70/100
zed-industries/zed#64680 · 2 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 2/5 1-3 小时 新手友好度 70/100
RustPython/RustPython#8802 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
TheLarkInn/aipm#2390 ·