Multi-second stall after a short blackout: PTO backoff (and occasional RTT sample spanning the blackout) delays recovery by up to one probe interval
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- rust
- Domain
- networking
Research direction
Read the PTO re-arming discussion in #841 and the path scheduling discussion in #516, then trace the QUIC PTO and congestion-control probe logic using the supplied timer_fired and PATH_ACK traces. The report asks whether configuration or an endpoint API can recover promptly after a blackout; a minimal reproduction is only available on request. Done means establishing the cause and a supported recovery approach that avoids delayed probing and misleading RTT samples.
Written by the indexing model from the issue text.
Description
Versions: iroh 1.3.0, noq 1.3.0 / noq-proto 1.3.0. Linux (Docker lab, netem).
Scenario: two endpoints on a direct path (NAT'd, hole-punched). One connection carries audio as 100 datagrams/s plus a reliable control stream. A total blackout of 5 s (netem loss 100% on both routers), after which the network comes back.
Expected: traffic resumes within ~1 RTT plus up to one PTO base after the network returns.
Observed (30 blackouts):
- 5 of them took 4.2–4.8 s to resume, while other connections on the same path resumed earlier.
- Traces (
timer_fired): the probe interval grows during the blackout: count 5 at −1.45 s, count 6 at +0.72 s, count 7 at +3.89 s (relative to the network coming back). Until the next probe, the path staysblocked by congestion control, with the cwnd full of packets sent during the blackout. - Sometimes the first ACK after the blackout acknowledges packets sent before the blackout (
PATH_ACK … delay: 915949µs). That yields an RTT sample about as long as the blackout, inflates srtt/rttvar, and pushes the PTO to 4–6 s (the connection stayed stuck for ~9 s in one trace).
What we tried (local patches, discarded):
- Skipping RTT samples whose largest acked packet was sent before the current PTO episode: no measurable improvement.
- Capping the absolute probe interval at
max_interval: recovery broke completely (the connection never resumed). We did not dig into why.
We also tried, at the application level, tearing down and re-establishing the stalled connection as soon as another connection to the same peer received data again. It made things worse (occasional long gaps), so we dropped it.
Question: is there a recommended configuration for short blackouts (e.g. Wi-Fi drops)? Or could the endpoint expose a way to reset the PTO backoff and probe immediately when any packet from the same remote arrives on another connection or path, which signals that the network is back? Endpoint::network_change() exists, but it seems aimed at interface changes and may abandon or reopen paths on the client.
Possibly related: #841 (PTO re-arming after a long stall) and #516 (switching scheduling to other paths faster).
Reproduction: the code is in a private repository, but we can share a minimal reproduction (Docker Compose with NAT'd routers and netem) and the traces on request.
- Dominant language
- Rust
- Stars
- 454
- Forks
- 98
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 19
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from n0-computer/noq
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
n0-computer/noq#892 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
n0-computer/noq#873 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
n0-computer/noq#872 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
n0-computer/noq#904 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 Half a day Newbie friendliness 68/100
n0-computer/noq#895 ·
Maintainers usually reply within 1 day
Similar issues
-
area:web kind:bug priority:p2 ready
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
Maintainers usually reply within 1 day
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 84/100
sims1253/r-typeshed#140 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
storytold/craft-libs#2867 ·
Maintainers usually reply within 1 day
-
help wanted needs investigation priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 62/100