[Bug]: selector-filtered _changes is slower the first time after creating a Mango index
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- erlang
- Domain
- databases
Research direction
Reproduce the slowdown with the provided curl commands using a large database, then compare the first and later selector-filtered _changes requests before and after creating the _index entry. Trace the _changes selector-filtering path alongside index creation or update handling. Done means the first filtered _changes request is not delayed by unrelated Mango indexes, with regression coverage for the reproduced case.
Written by the indexing model from the issue text.
Description
Version
3.5.1, and main branch
Describe the problem you're encountering
Despite the fact that _changes does not use indexes to speed up selector filters, the first time it is requested after creating a Mango index, it is slower, which possibly indicates it's spending time updating the index.
Expected Behaviour
The time taken for _changes to respond should not depend on the existence or otherwise of Mango indexes.
Steps to Reproduce
To demonstrate this, I created a DB containing a million docs of the form { "_id": "doc-N", "value": N } for N from 1 to 1,000,000. Then I timed how long it took to run this filtered changes request:
# uses this shell function:
cdb () {
curl -gsu 'admin:admin' "http://127.0.0.1:5984$1" "${@:2}"
}
time ( cdb '/db/_changes?filter=_selector' -X POST -d '{ "selector": { "value": { "$lt": 100 } } }' )
This would take around 12 seconds.
Then I created an index covering the filtered field and re-timed:
cdb '/db/_index' -X POST -d '{ "index": { "fields": ["value"] }, "name": "by-value", "type": "json" }'
time ( cdb '/db/_changes?filter=_selector' -X POST -d '{ "selector": { "value": { "$lt": 100 } } }' )
The first time after creating an index, _changes takes over 30s to complete. On later requests it's back to around 12s, possibly very slightly slower.
This effect shows up regardless of which fields the index covers; even if it does not cover the _changes selector, the _changes request is still slow.
Your Environment
macOS 15, CouchDB 3.5.1, single node, database created with q=2, n=1
Additional Context
No response
- Dominant language
- Erlang
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 1d 23m
- Merged PRs (30d)
- 30
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/couchdb
-
Fix: Warning: 'catch ...' is deprecated; please use 'try ... catch ... end' insteadPossibly taken @devx-arjun claimed this 4 days ago. Openbeginner-friendly build chore patches-welcome
Difficulty 4/5 3-5 days Newbie friendliness 45/100
apache/couchdb#6144 · 4 comments ·
Maintainers usually reply within 1 day
-
bug needs-triage
Difficulty 4/5 3-5 days Newbie friendliness 48/100
apache/couchdb#6132 · 3 comments ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
apache/couchdb#6120 · 4 comments ·
Maintainers usually reply within 1 day
-
enhancement needs-triage
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Maintainers usually reply within 1 day
-
bug needs-triage
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
Similar issues
-
Add: CartoonitoOpencheck:failed feeds:add
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
iptv-org/database#37390 · 1 comment ·
Maintainers usually reply within 9 days
-
[BUG] LazyStackedTensorDictStore zeroes the last byte of a new key set on the last elementPossibly taken @peterdsharpe claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
pytorch/tensordict#2307 ·
Maintainers usually reply within 1 day
-
Add Group doesn't trim the group name, so a spaces-only name creates a blank group and "fossy " bypasses the duplicate checkPossibly taken @bhuvan-somisetty claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
fossology/fossology#3917 · 1 comment ·
Maintainers usually reply within 2 days
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
area: sql provider: sybase
Difficulty 1/5 Under an hour Newbie friendliness 83/100
Maintainers usually reply within 1 day