Product quantization (PQ) baseline in the lab
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 65/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- databases, machine-learning
Research direction
Start with the codec implementations in src/shrinkvec/codecs/ and follow TurboQuant as the model. Read tests/test_crosscheck.py to see how recall is compared against faiss IndexPQ. Done means pq-<m>x<bits> works in the lab, byte counts include shared codebooks, and the cross-check passes.
Written by the indexing model from the issue text.
Description
PQ is the classic baseline the TurboQuant and RaBitQ papers compare against, and it trains on the data. Having it in the lab shows what training buys.
Where to start: a new codec in src/shrinkvec/codecs/ following TurboQuant. Cross-check recall against faiss IndexPQ in tests/test_crosscheck.py.
Done when: pq-<m>x<bits> works in the lab, byte counts include the codebooks as shared state, and the cross-check passes.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 35m
- Merged PRs (30d)
- 5
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ahmad-codex/shrinkvec
-
area: report enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
ahmad-codex/shrinkvec#15 ·
-
area: report enhancement
Difficulty 4/5 3-5 days Newbie friendliness 56/100
ahmad-codex/shrinkvec#16 ·
-
Faster lab scansOpenarea: lab help wanted performance
Difficulty 4/5 3-5 days Newbie friendliness 42/100
ahmad-codex/shrinkvec#14 ·
-
enhancement good first issue performance
Difficulty 3/5 1-2 days Newbie friendliness 72/100
ahmad-codex/shrinkvec#12 ·
-
area: lab codec enhancement
Difficulty 4/5 3-5 days Newbie friendliness 56/100
ahmad-codex/shrinkvec#11 ·
All issues in ahmad-codex/shrinkvec
Similar issues
-
follow-up
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
modelscope/modelscope#1833 ·
Maintainers usually reply within 1 day
-
ocr_data_to_elements() raises AttributeError when a layout element is a ListPossibly taken @Eapp1e claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
Unstructured-IO/unstructured#4616 ·
Maintainers usually reply within 1 day