[Bug]: DELETE /{db} returns 500 badarg instead of 404 when the shard map cache still lists a just-deleted database
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 90/100
Research direction
Start in src/fabric/src/fabric_db_delete.erl at fabric_db_delete:maybe_stop/2, specifically the branch where every shard returns not_found. Check the warning format and verify the completed behavior with the reported concurrent-reader reproduction: the second DELETE returns 404 with not_found and the warning is logged instead of producing a 500 badarg.
Written by the indexing model from the issue text.
Description
Version
3.5.1
Describe the problem you're encountering
Right after a database is deleted, a second DELETE /{db} can answer 500 {"error":"unknown_error","reason":"badarg"} instead of 404 not_found. The stack trace in the log points to fabric_db_delete:maybe_stop/2.
That branch should turn "every shard answered not_found" into {stop, not_found}, but it crashes while formatting its own log message:
%% src/fabric/src/fabric_db_delete.erl (line 79 in 3.5.1, line 78 in 3.5.2 and main)
{W, 0, W} ->
{#shard{dbname = Name}, _} = hd(Counters),
couch_log:warning("~p not_found ~d", [?MODULE, Name]),
{stop, not_found};
Name is a binary, so ~d raises badarg and the request fails with a 500.
This branch is only reached when the shard map cache (mem3) still lists the database after it was deleted, for example when a read that raced with the deletion put its shards back in the cache. Until the cache catches up, other requests on that database also fail with 500 "No DB shards could be opened.". That part may be expected while the deletion propagates; the badarg is not.
Expected Behaviour
The second DELETE should answer 404 {"error":"not_found","reason":"Database does not exist."}, and the warning should be logged. Using ~s (or ~p) instead of ~d in that format string should be enough.
Steps to Reproduce
On a single-node CouchDB 3.5.1 (couchdb:3.5.1 Docker image, default config). Replace $COUCH with the server URL including admin credentials.
DB=badarg_repro
curl -s -X PUT "$COUCH/$DB"
# a few concurrent readers
for i in 1 2 3 4; do
( while true; do curl -s -o /dev/null "$COUCH/$DB"; done ) &
done
curl -s -X DELETE "$COUCH/$DB" # {"ok":true}
curl -s -X DELETE "$COUCH/$DB" # often: 500 {"error":"unknown_error","reason":"badarg","ref":...}
kill %1 %2 %3 %4
With this script, the second DELETE answered 500 badarg (always "ref":17476293) in 14 of 40 runs on our machine; a tighter loop hit it in 188 of 200. With no concurrent readers it did not reproduce (0 out of 150).
Your Environment
- CouchDB 3.5.1, official Docker image, single node
- Seen from the CouchDB integration tests of rouchdb, on a GitHub Actions Linux runner and on Docker Desktop for macOS
Additional Context
The format string is the same in 3.5.2 and on main, so those are likely affected as well. On the client side we now retry these 500s for about a second after a delete.
- Dominant language
- Erlang
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 18
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/couchdb
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 55/100
apache/couchdb#6139 · 1 comment ·
Maintainers usually reply within 1 day
-
bug needs-triage
Difficulty 4/5 3-5 days Newbie friendliness 48/100
apache/couchdb#6132 · 3 comments ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
apache/couchdb#6120 · 4 comments ·
Maintainers usually reply within 1 day
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
apache/couchdb#6118 · 7 comments ·
Maintainers usually reply within 1 day
-
enhancement needs-triage
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
collective/icalendar#1854 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
rancher/rancher-ai-agent#412 ·
Maintainers usually reply within 6 days
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
apache/arrow-java#1311 ·
Maintainers usually reply within 2 days
-
Mend: dependency security vulnerability
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
opentok/Opentok-Python-SDK#272 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
NVIDIA/earth2studio#1203 ·
Maintainers usually reply within 2 days