[Bug]: Ghost view rows persisting after their corresponding docs have been purged
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 40/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- erlang, javascript
Research direction
The issue includes a detailed reproduction script. Start by running the script to confirm the ghost rows. Examine the view indexing logic in the CouchDB source, likely in the view server or indexer modules. Look for how purges interact with view updates. The goal is to understand why rows persist after purges and ensure they are cleaned up during normal operations.
Written by the indexing model from the issue text.
Description
Version
3.5.2
Describe the problem you're encountering
I am seeing view row ghosts when I purge large amounts of documents, while querying the view in between purge batches. These ghost rows return doc: null when queried with include_docs=true, and they never go away, they survive new docs being pushed and compaction. Only a full index rebuild seems to remove them.
The view reports that it caught up with the purges.
Expected Behaviour
I would expect the view to never return rows with no docs, or at least to heal after some point.
Steps to Reproduce
Repro script available here: https://gist.github.com/dianabarsan/5e21753d6545e6f61429ed31ebe79714
- create throwaway db
- create one design doc and one view that would index every doc
- seed db with 300k documents
- with a batch size of 1000, purge all documents, querying the view after each batch
- after all docs are purged, query the view and inspect the results
Sample result:
CouchDb 3.5.2, 300000 docs, purging 1000 at a time, querying view after each batch
attempt 1: 543 ghost view rows
db doc_count 0
db purge_seq 300000-g1AAAAIdeJx90C0OwkAQBeAJIDEILoEgFIOEGxBo5wD9Ec2mgYQEjeIIlVhoh-BxgOAACDgEZ6BdZqd-dsUT871NZjMA6KbtBHrxehunSTTzxpPhiK-X8ai16sCJ6Mth-ZgIIHzqfIn45jhyS_he55W1N44Ft4RvdF4QEcePW8LnOvcR82YFEn5ReLSztm4WyAXfNWyISteohL40ioiB84XQq0an_MnO-0LPGh3wTs7XQg8a7fOLzpdCPwoNweFA3MP8AXUdop0
view purge_seq 300000
view query result:
{
total_rows: 543,
offset: 0,
rows: [
{ id: 'd32020', key: 'd32020', value: null, doc: null },
{ id: 'd32024', key: 'd32024', value: null, doc: null },
{ id: 'd32028', key: 'd32028', value: null, doc: null },
{ id: 'd32034', key: 'd32034', value: null, doc: null },
{ id: 'd32069', key: 'd32069', value: null, doc: null },
{ id: 'd32071', key: 'd32071', value: null, doc: null },
{ id: 'd32075', key: 'd32075', value: null, doc: null },
{ id: 'd32079', key: 'd32079', value: null, doc: null },
{ id: 'd32092', key: 'd32092', value: null, doc: null },
{ id: 'd32096', key: 'd32096', value: null, doc: null },
... 533 more items
]
}
What I see in Fauxton:
https://github.com/user-attachments/assets/caec4929-ebe1-4910-bc5b-d566e31ad6a4
Your Environment
Manjaro Linux, CouchDb 5.3.2 in Docker
Additional Context
No response
- Dominant language
- Erlang
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 5h 24m
- Merged PRs (30d)
- 10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/couchdb
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 40/100
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
enhancement needs-triage
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
bug needs-triage
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
mksglu/context-mode#1200 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
clawsweeper:needs-maintainer-review clawsweeper:needs-product-decision clawsweeper:no-new-fix-pr impact:auth-provider issue-rating: 🌊 off-meta tidepool P2
Difficulty 1/5 Under an hour Newbie friendliness 80/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100