[Telemetry Data] May 14 ZKSync Sequencer Stall: 5000ms TCP Timeout + Block Production 10x Below Normal (Public Status: Green)
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 15/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- blockchain
- Domain
- blockchain, observability-sre
Research direction
No repository file, test, or entry point is named, and the issue does not specify a repository change. Start by comparing the report with issue #140 and validating the cited telemetry and on-chain observations; completion criteria are not defined.
Written by the indexing model from the issue text.
Description
[Telemetry Data] May 14 ZKSync Sequencer Stall: 5000ms TCP Timeout + Block Production 10x Below Normal (Public Status: Green)
Same monitoring infrastructure that captured the May 13 Arbitrum stall fired twice today on ZKSync Era.
What Phoenix Zero Recorded
EVENT 1 — 13:57 UTC
🟣 SEQUENCER STALL — ZKSYNC
RTT: 5000ms | P99: 5000ms | P95: 5000ms
EVENT 2 — 14:05 UTC
🟣 SEQUENCER STALL — ZKSYNC
RTT: 5000ms | P99: 5000ms | P95: 5000ms
5000ms = hard socket timeout. The sequencer did not respond to TCP packets within 5 seconds. For an HFT router, this is a black hole: transactions submitted in this window are dropped at the transport layer, never reaching a block.
On-Chain Verification — Block Production Rate
Normal ZKSync Era block rate: ~2 seconds/block.
| Block range | Time span | Rate | vs normal |
|---|---|---|---|
| 70098220 → 70098230 | 13:56:25 → 13:57:01 UTC | 3.6s/block | baseline |
| 70098240 → 70098250 | 13:57:26 → 13:58:17 UTC | 5.1s/block | 2.5x slower |
| 70098310 → 70098320 | 14:00:57 → 14:02:10 UTC | 7.3s/block | 3.6x slower |
| 70098340 → 70098345 | 14:03:22 → 14:04:13 UTC | 10.2s/block | 5x slower ← peak |
Block production did not stop entirely — but throughput dropped to 1 tx/block (vs normal 2-4) during the stall window. The RPC layer was returning 5000ms timeouts while the sequencer was producing skeleton blocks at severely degraded rate.
Public status pages: no incident reported.
Concurrent Events (Same Window)
| Chain | Time (UTC) | Reading | Type |
|---|---|---|---|
| Optimism | 13:54–14:04 | P99: 413–574ms sustained | 🔵 SLOW CREEP |
| Base | 14:00 | P99: 2190ms spike | 🔵 SLOW CREEP |
| ZKSync | 13:57, 14:05 | RTT: 5000ms | 🟣 SEQUENCER STALL |
Multi-chain degradation in the same 10-minute window. Base recovered immediately; Optimism sustained degradation for 10 minutes; ZKSync hit the hard timeout ceiling twice.
Context
ZKSync Era's proof system currently has a documented partial liveness vulnerability (source: L2BEAT). Today's 5000ms TCP timeouts are consistent with this ongoing issue manifesting under load.
The block explorer API itself lags ~2.5 hours behind real-time — meaning on-chain verification of the exact stall window requires waiting for indexing. The RTT sensor fires within 2 seconds of the event.
This is the delta: public infrastructure reports green. Our transport-layer sensor sees the black hole before the block explorer indexes it.
Infrastructure
Phoenix Zero RTT Oracle probes 4 chains (Base, Arbitrum, Optimism, ZKSync) via eth_blockNumber every 2 seconds.
- WebSocket stream:
wss://rtt.phoenix-ai.work/ws - Live Telegram alerts:
@Phoenix_Node_Alex_Bot - Server: DigitalOcean NYC1 (198.211.103.36)
Previous data point: May 13 Arbitrum stall — 3081ms, 47s delta vs public RPC
Contact: aleksandrkent64@gmail.com
- Dominant language
- No language data
- Stars
- 2.6k
- Forks
- 291
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from flashbots/pm
-
[Telemetry Data] May 13 Arbitrum Sequencer Stall: eBPF Kernel Reality vs Public RPC (47s Delta) Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
crypto Open
Difficulty 5/5 Over a week Newbie friendliness 10/100
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
MystenLabs/sui#28056 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
filecoin-project/solstice#76 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
CypherBoxLLC/Cypher-Box#283 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·