[Bug]: View requests fail with a 500 compaction error if queried in between purges

Start with src/couch_mrview_compactor.erl at swap_compacted/2, then trace the commit_compacted call in src/couch_index.erl. Run the linked purge-and-query reproduction script against CouchDB main and observe the purge/view-compaction race. Done means repeated view requests during batched purges no longer return the 500 enoent compaction error.

Version

3.5.2

Describe the problem you're encountering

I am seeing this error sporadically (6 out of 3000) when I am doing mass purging and querying a view between purge batches:

GET /purge-view-ghosts-1790228958432/_design/d/_view/v?limit=1
{badmatch,{error,enoent}} | ref 4180022058 | couch_mrview_compactor:swap_compacted/2 (src/couch_mrview_compactor.erl:258)
[{couch_mrview_compactor,swap_compacted,2, [{file,"src/couch_mrview_compactor.erl"},{line,258}]}, {couch_index,commit_compacted,2,[{file,"src/couch_index.erl"},{line,393}]}, {couch_index,handle_call,3,[{file,"src/couch_index.erl"},{line,190}]}, {gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,1131}]}, {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,1160}]}, {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,241}]}]

Expected Behaviour

View requests should succeed.

Steps to Reproduce

This repro script can be used: https://gist.github.com/dianabarsan/5e21753d6545e6f61429ed31ebe79714

Steps:

• create throwaway db
• create one design doc and one view that would index every doc
• seed db with 300k documents
• with a batch size of 1000, purge all documents, querying the view after each batch
• log any view query errors

Sample result:

attempt 1: 0 ghost view rows, retrying
attempt 2: 0 ghost view rows, retrying
Compaction error during view query
Error: GET /purge-view-ghosts-1790235371583/_design/d/_view/v?limit=1 -> 500 {"error":"{badmatch,{error,enoent}}","reason":"[{couch_mrview_compactor,swap_compacted,2,\n                         [{file,\"src/couch_mrview_compactor.erl\"},{line,267}]},\n {couch_index,commit_compacted,2,[{file,\"src/couch_index.erl\"},{line,393}]},\n {couch_index,handle_call,3,[{file,\"src/couch_index.erl\"},{line,190}]},\n {gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,2470}]},\n {gen_server,handle_msg,3,[{file,\"gen_server.erl\"},{line,2499}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,333}]}]","ref":106889267}
    at async /home/.../scratches/couch-purge-view-ghosts-minimal.js:106:5
attempt 3: 0 ghost view rows, retrying
attempt 4: 0 ghost view rows, retrying
attempt 5: 0 ghost view rows, retrying
Compaction error during view query
Error: GET /purge-view-ghosts-1790235494953/_design/d/_view/v?limit=1 -> 500 {"error":"{badmatch,{error,enoent}}","reason":"[{couch_mrview_compactor,swap_compacted,2,\n                         [{file,\"src/couch_mrview_compactor.erl\"},{line,267}]},\n {couch_index,commit_compacted,2,[{file,\"src/couch_index.erl\"},{line,393}]},\n {couch_index,handle_call,3,[{file,\"src/couch_index.erl\"},{line,190}]},\n {gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,2470}]},\n {gen_server,handle_msg,3,[{file,\"gen_server.erl\"},{line,2499}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,333}]}]","ref":106889267}
       at async /home/.../scratches/couch-purge-view-ghosts-minimal.js:106:5
attempt 6: 0 ghost view rows, retrying
Compaction error during view query
Error: GET /purge-view-ghosts-1790235537142/_design/d/_view/v?limit=1 -> 500 {"error":"{badmatch,{error,enoent}}","reason":"[{couch_mrview_compactor,swap_compacted,2,\n                         [{file,\"src/couch_mrview_compactor.erl\"},{line,267}]},\n {couch_index,commit_compacted,2,[{file,\"src/couch_index.erl\"},{line,393}]},\n {couch_index,handle_call,3,[{file,\"src/couch_index.erl\"},{line,190}]},\n {gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,2470}]},\n {gen_server,handle_msg,3,[{file,\"gen_server.erl\"},{line,2499}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,333}]}]","ref":106889267}
        at async /home/.../scratches/couch-purge-view-ghosts-minimal.js:106:5
attempt 7: 0 ghost view rows, retrying
Compaction error during view query
Error: GET /purge-view-ghosts-1790235581055/_design/d/_view/v?limit=1 -> 500 {"error":"{badmatch,{error,enoent}}","reason":"[{couch_mrview_compactor,swap_compacted,2,\n                         [{file,\"src/couch_mrview_compactor.erl\"},{line,267}]},\n {couch_index,commit_compacted,2,[{file,\"src/couch_index.erl\"},{line,393}]},\n {couch_index,handle_call,3,[{file,\"src/couch_index.erl\"},{line,190}]},\n {gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,2470}]},\n {gen_server,handle_msg,3,[{file,\"gen_server.erl\"},{line,2499}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,333}]}]","ref":106889267}
        at async /home/.../scratches/couch-purge-view-ghosts-minimal.js:106:5
attempt 8: 0 ghost view rows, retrying
Compaction error during view query
Error: GET /purge-view-ghosts-1790235628307/_design/d/_view/v?limit=1 -> 500 {"error":"{badmatch,{error,enoent}}","reason":"[{couch_mrview_compactor,swap_compacted,2,\n                         [{file,\"src/couch_mrview_compactor.erl\"},{line,267}]},\n {couch_index,commit_compacted,2,[{file,\"src/couch_index.erl\"},{line,393}]},\n {couch_index,handle_call,3,[{file,\"src/couch_index.erl\"},{line,190}]},\n {gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,2470}]},\n {gen_server,handle_msg,3,[{file,\"gen_server.erl\"},{line,2499}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,333}]}]","ref":106889267}
        at async /home/.../scratches/couch-purge-view-ghosts-minimal.js:106:5
attempt 9: 0 ghost view rows, retrying
Compaction error during view query
Error: GET /purge-view-ghosts-1790235673510/_design/d/_view/v?limit=1 -> 500 {"error":"{badmatch,{error,enoent}}","reason":"[{couch_mrview_compactor,swap_compacted,2,\n                         [{file,\"src/couch_mrview_compactor.erl\"},{line,267}]},\n {couch_index,commit_compacted,2,[{file,\"src/couch_index.erl\"},{line,393}]},\n {couch_index,handle_call,3,[{file,\"src/couch_index.erl\"},{line,190}]},\n {gen_server,try_handle_call,4,[{file,\"gen_server.erl\"},{line,2470}]},\n {gen_server,handle_msg,3,[{file,\"gen_server.erl\"},{line,2499}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,333}]}]","ref":106889267}
        at async /home/.../scratches/couch-purge-view-ghosts-minimal.js:106:5
attempt 10: 0 ghost view rows, retrying
no ghost rows in 10 attempts

Your Environment

Manjaro Linux, CouchDB main ~ 3.5.2 (to include this commit https://github.com/apache/couchdb/commit/1582cf98c5ac97f701c809ee2668026ff8a9a5b1) in Docker

Additional Context

Branched out of https://github.com/apache/couchdb/issues/6128#issuecomment-5818101233