A loaded-model tier: EmbeddingGemma added to the calibrated decision
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- ai-infra-agents, backend
Research direction
Read REPORT §4w and receipts bench-20261026-heldout-dense-embeddinggemma-words.txt, then inspect the prompt hook and MCP server entry points to understand how they start and communicate. Build a production implementation that keeps EmbeddingGemma loaded, then run a pre-registered evaluation of judged precision and recall on real prompts. Done means the production path and real-prompt evaluation settle whether the tier should ship.
Written by the indexing model from the issue text.
Description
REPORT §4w's exploratory arm, shared subject words + 8 × EmbeddingGemma-300m's best cosine (model-card prompts), reached 95.2% [93.0, 97.4] held-out recall against 90.0% for the model-free decision, most of it on paraphrases (73 → 81 of 90), with 12 of 268 subject-less prompts briefed against 9 (receipt bench-20261026-heldout-dense-embeddinggemma-words.txt). A prompt hook is a new process, and the model costs 1.7–2.1 s to load there (12.4 s on a first run), so it needs a process that keeps it loaded: the MCP server, or a small daemon the hook talks to.
To settle before it ships: a production implementation, then judged precision and recall on real prompts (the bench's synthetic queries can flatter it), pre-registered. Refs #291, #292.
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 5h 10m
- Merged PRs (30d)
- 127
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from emeraldleaf/okl
-
enhancement loop-review
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
emeraldleaf/okl#190 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
emeraldleaf/okl#147 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 50/100
emeraldleaf/okl#356 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
emeraldleaf/okl#349 ·
Maintainers usually reply within 1 day
-
okl's own CI against the from-scratch guide: Sonar coverage, format gate, locked installsPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
emeraldleaf/okl#331 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
elviskahoro/sdk-python-attio#144 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
RenyiELBO raises TypeError when model has no sample sitesPossibly taken @alibb007 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day