search/find ranking: generic entities outrank project-specific memories (golden-query baseline)

Open
  • Is something wrong?
#7,331 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
ai, search

Research direction

Start by tracing the POST /api/v1/search/find ranking path and checking whether configuration already exposes a relevance or category-normalization option; the issue does not name implementation files or tests. Use the reported queries, open-bot deployment and plugin page server, as baseline cases, and compare top results against the open-bot golden-query harness at scripts/ov-eval.ts and scripts/ov-eval.golden.json. Done means project-specific memories rank ahead of unrelated generic entities while sorted and stable result checks still pass.

Written by the indexing model from the issue text.

Description

Summary

Semantic search/find ranking: generic provider entities outrank project-specific memories for project queries. Reporting with a repeatable golden-query baseline from a production open-bot instance (per-user OpenViking sidecar).

Baseline (2026-11-10, POST /api/v1/search/find, limit 5)

Query: open-bot deployment

  1. viking://user/agent/memories/entities/api/openai.md — 0.895 (generic API-provider entity)
  2. .../entities/promotion/openbot_promotion.md — 0.883
  3. viking://user/agent/memories/entities/modele_ia/clef.md — 0.865 (unrelated model entity)
  4. .../entities/promotion/openbot_promo_video.md — 0.862

Query: plugin page server

  1. .../events/2026/11/04/Chargement des plugins activés.md — 0.829
  2. .../events/2026/10/16/Migration et tests du plugin api-connectors.md — 0.828

Hard properties look good: scores are sorted descending and results are stable across repeated identical queries. The gap is relevance ordering: high-centrality generic entities (OpenAI, an unrelated model) beat memories that literally concern the queried project.

Harness

A repeatable golden-query eval lives in open-bot as scripts/ov-eval.ts + scripts/ov-eval.golden.json (records top-5 URIs/scores, latency, sorted/stable checks per query). Happy to extend the golden set or re-run against any candidate fix and share before/after.

Ask

Guidance or ranking work so project/topic-specific memories outrank generic entity pages for specific queries (e.g. lexical/entity-name match boost, or per-category score normalization). If a configuration knob already exists for this, pointing at it closes this just as well.

Related downstream reports (open-bot): shpaw415/open-bot#20, #21, #32, #71, #83.

Dominant language
Python
Stars
41.8k
Forks
3.4k
Avg merge
19h 29m
Merged PRs (30d)
500

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from volcengine/OpenViking

All issues in volcengine/OpenViking

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.