BLPOP with a 0 timeout queued in MULTI never replies at EXEC, and leaves the key unusable for every client
Maintainers usually reply within 2 days
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 55/100
Research direction
The issue identifies BLPOP inside MULTI/EXEC as the failing path, but names no source files or tests. Start by tracing how Garnet executes blocking list commands inside transactions and find the existing tests for empty-list BLPOP with a zero timeout. Reproduce the hang and verify that EXEC returns [nil] and a fresh connection can use the key afterward; check the other listed blocking commands for the same behavior.
Written by the indexing model from the issue text.
Description
Describe the bug
BLPOP key 0 queued inside MULTI and then run by EXEC, on an empty or missing list, never
produces a reply. The client is stuck, and so is the key: a later write to that key from a fresh
connection also never replies. Unrelated keys keep working, and FLUSHALL does not help — only
restarting the process does.
DEL k1
MULTI -> OK
BLPOP k1 0 -> QUEUED
EXEC -> (no reply, ever)
PING -> (no reply; the connection is unusable)
then, from a second, brand-new connection:
PING -> PONG the server is alive
SET unrelated 1 -> OK other keys are fine
LPUSH k1 y -> (no reply, ever)
So one client that issues a queued BLPOP ... 0 permanently removes a key from service for
everybody, and burns a connection each time something touches it. Reaching it needs no special
privilege — a single ordinary transaction does it — which makes it a denial of service on any key an
untrusted client can name.
It is specific to the blocking-forever case inside a transaction. All of these are fine:
| variant | Garnet |
|---|---|
MULTI; BLPOP k 1; EXEC (finite timeout) |
nil ✅ |
MULTI; LPOP k; EXEC |
nil ✅ |
BLPOP k 0 outside MULTI, list non-empty |
returns the element ✅ |
MULTI; BLPOP k 0; EXEC, list non-empty |
returns the element ✅ |
MULTI; BLPOP k 0; EXEC, list empty |
hangs forever ❌ |
Steps to reproduce the bug
- Start Garnet 2.2.2:
docker run -d -p 6400:6379 ghcr.io/microsoft/garnet:2.2.2 - On one connection, from a clean keyspace (
FLUSHALL):
DEL k1
MULTI
BLPOP k1 0
EXEC <- no reply arrives; the connection is now stuck
- Open a second connection and try to use that key:
PING -> PONG
LPUSH k1 y <- no reply arrives either
FLUSHALLfrom a third connection does not restore the key; restarting the process does.
Each row below is an independent sequence from a clean keyspace, each on its own key (sharing one
key would make the controls look broken too, since the first case wedges it). "no reply" means
nothing arrived within the client timeout.
| case | expected | Redis 8.10.2 | Kvrocks | KeyDB | Pika | Dragonfly | Garnet 2.2.2 |
|---|---|---|---|---|---|---|---|
MULTI; BLPOP k1 0; EXEC |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
no reply |
… then PING on the same connection |
PONG |
PONG |
PONG |
PONG |
PONG |
PONG |
no reply |
MULTI; BLPOP k3 1; EXEC (finite timeout) |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
MULTI; LPOP k4; EXEC |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
[nil] |
BLPOP k5 0 outside MULTI, non-empty list |
[k5, x] |
[k5, x] |
[k5, x] |
[k5, x] |
[k5, x] |
[k5, x] |
[k5, x] |
MULTI; BLPOP k6 0; EXEC, non-empty list |
[[k6, x]] |
[[k6, x]] |
[[k6, x]] |
[[k6, x]] |
[[k6, x]] |
[[k6, x]] |
[[k6, x]] |
afterwards, fresh connection: LPUSH k1 y |
1 |
1 |
1 |
1 |
1 |
1 |
no reply |
A self-contained Python 3 reproducer (standard library only) that runs every case, prints this table
and then probes the wedged key from fresh connections is attached as repro.py; it exits 0 when the
bug reproduces.
Expected behavior
EXEC should reply [nil], as every other implementation here does. The BLPOP documentation
specifies this case and gives exactly the reason it must not block — it would hold up the server:
BLPOPinside aMULTI/EXECtransactionUsing
BLPOPinside a MULTI / EXEC block does not make a lot of sense as it would require
blocking the entire server in order to execute the block atomically, which in turn does not allow
other clients to perform a push operation. For this reason the behavior ofBLPOPinside
MULTI / EXEC when the list is empty is to return anilmulti-bulk reply, which is the same
thing that happens when the timeout is reached.If you like science fiction, think of time flowing at infinite speed inside a MULTI / EXEC block...
The same applies to the other blocking list and sorted-set commands (BRPOP, BLMOVE,
BRPOPLPUSH, BLMPOP, BZPOPMIN, BZPOPMAX, BZMPOP) — we only verified BLPOP, but they are
worth checking together.
Screenshots
No response
Release version
v2.2.2
IDE
No response
OS version
Ubuntu 24.04.5 LTS, kernel 7.0.0-38-generic (x86_64), Docker. The container reports
os:Unix 7.0.0.38.
Additional context
Environment where the bug was reproduced contains Garnet 2.2.2, from the published container image ghcr.io/microsoft/garnet:2.2.2
(digest sha256:536185d34d4783856401f79f85d98c0cd46b417fa1a1c2ddd4441bc4012c4f34).
INFO server reports garnet_version:2.2.2. Source read at
456925d8b36eb139d0a37937b85c085ab815c482 (tag v2.2.2).
- Dominant language
- C#
- Stars
- 12.1k
- Forks
- 716
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 65
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/garnet
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 2 days
-
GEOADD updating a member's position keeps the member's old TTL, so the new position is silently deletedMay be free again @Saad-elnagar claimed this 8 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
microsoft/garnet#2265 · 1 comment ·
Maintainers usually reply within 2 days
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 2 days
All issues in microsoft/garnet
Similar issues
-
area: core priority: low type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
RicoSuter/Namotion.Interceptor#863 ·
Maintainers usually reply within 1 day
-
area-cli
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
0.3.0
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
[Bug] Decompression leaves metadata that makes recompression throw and corrupt the content typeOpen
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
BrighterCommand/Brighter#4683 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100