hyper-util legacy client: an HTTP/1 request can hang forever when the connection closes while the request is being queued
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 62/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- rust
调研方向
Start at hyper_util::client::legacy::Client::try_send_request and PoolClient::poll_ready, then inspect the HTTP/1 dispatcher close path described in the issue. Use the deterministic regression tests in the linked hyper-util patch tree as the starting test reference. Done means a dispatcher close releases the held sender and the request completes with the existing retry or cancellation behavior instead of hanging.
由索引模型根据 Issue 内容生成。
描述
Summary
hyper_util::client::legacy::Client can leave an HTTP/1 request future pending
forever. This happens when the pooled connection's dispatcher closes (the peer
sends RST, or a FIN on an idle keep-alive connection) at the moment the request
is being enqueued. No error is returned and the request is not retried. The
caller sees a hang that ends only when its own timeout fires.
Versions: hyper 1.9.0, hyper-util 0.1.20, tokio 1.52.3. hyper and hyper-util
master have the same code.
Mechanism
Client::try_send_requestchecks out an HTTP/1Pooled<PoolClient>and
callspooled.try_send_request(req).await. It holdspooledacross the
await.hyper::client::dispatch::Sender::try_sendcalls tokio's
UnboundedSender::send, which works in two steps:
inc_num_messages()checks the closed bit and reserves a slot, then
chan.send()publishes the value.- Suppose the requester is preempted between those two steps while the
connection task sees the peer's reset.proto::h1::dispatch::Client::recv_msg(Err)
finds no callback. It callsrx.close(), and thenrx.try_recv(). tokio's
recvreturnsPendingwhile a send is in progress (the semaphore is not
idle), andtry_recvusesnow_or_never, so it returnsNone. The
dispatcher returnsErr, and hyper-util logs
client connection error: ... ConnectionReset. On a graceful FIN, the
dispatcher finishesOkinstead and drops its receiver. - Dropping
UnboundedReceiverdrains only published values. - The requester resumes and publishes into a channel nobody will read.
Envelope::dropwould fail the callback withCanceledand give back the
request. But it runs only when the channel is dropped, and the channel
stays alive whilepooledholds its only sender.pooledis held until the
response arrives, and the response never arrives.
tokio documents that a value can be sent after the receiver is dropped
(tokio-rs/tokio#7714, tokio-rs/tokio#8278).
Evidence
In CI, a server that accepts a connection and immediately resets it
(SO_LINGER=0) produces this trace:
hyper_util::client::legacy::client: connection is ready
hyper_util::client::legacy::pool: checkout dropped
<server accepts, resets>
hyper_util::client::legacy::client: client connection error: hyper::Error(Io, Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })
hyper_util::client::legacy::client: sending connection error to error channel
<nothing until the caller's 5 s timeout>
The client connection error line appears only when the dispatcher had no
callback and try_recv found nothing. Every other ordering fails the request
in microseconds. The hang needs a preemption inside a window of a few
instructions, so it shows up only on loaded, many-threaded runners. We saw it
five times in eight days of CI across nine variants of that test, and never in
local repetition.
Proposed fix (hyper-util)
Do not keep holding the only HTTP/1 sender after its dispatcher has closed.
While awaiting the response, try_send_request also polls
PoolClient::poll_ready for HTTP/1 connections:
Err(thewanttaker was canceled, which happens onrx.close()and on
receiver drop): captureis_reused()andconn_info, then droppooled.
That drops the last sender. The channel destructor then drops any stranded
envelope, and hyper fails the callback withCanceledplus the request. The
existingRetryablepath then retries a reused connection or returns
Canceledfor a fresh one, which is exactly what happens when the dispatcher
does see the queued request.Ok(the dispatcher asked for more work after the request was published):
it saw the request, so stop watching.
The response future is polled first on every wake, so a delivered response or
error always wins. The change does not allocate and changes no API.
The patch we currently carry against hyper-util 0.1.20 is here:
https://github.com/ferrum-edge/ferrum-edge/blob/d1136ab60f0c74221bae49cc224b265bdcb24575/docs/upstream-hyper-util-patches/001-release-h1-sender-on-dispatch-close/hyper-util-release-h1-sender-on-dispatch-close.patch
(with deterministic regression tests in the same vendored tree). We are happy to open a PR if this direction looks right.
An alternative fix in hyper: after rx.close(), keep the dispatcher polling
rx until it returns Ready(None), and cancel anything that arrives. Do the
same before dropping the receiver on a graceful shutdown. That fixes every
SendRequest user, including code that drives client::conn::http1 directly
and holds its sender while waiting. But it keeps the connection task alive
after close until the preempted sender runs again.
- 主要语言
- Rust
- 星标
- 16.3k
- 派生
- 1.8k
- 平均合并
- 2 天 20 小时
- 30 天内合并 PR
- 14
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
hyperium/hyper 的其他 Issue
-
C-feature
难度 1/5 1 小时以内 新手友好度 65/100
-
C-feature
难度 5/5 一周以上 新手友好度 35/100
-
C-bug S-waiting-on-author
-
A-http2 C-bug
-
C-bug
难度 4/5 3-5 天 新手友好度 48/100
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
BurntSushi/jiff#653 ·
-
难度 2/5 1-3 小时 新手友好度 72/100
ZcashFoundation/zeeder#106 ·
-
难度 2/5 1-3 小时 新手友好度 76/100
oxidize-rb/rb-sys#807 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
-
todo:ticket
难度 2/5 1-2 天 新手友好度 74/100