RFC: Optional local EmbeddingGemma 2 backend for semantic search

This is a design RFC, not a task: it asks maintainers whether an optional supervised Python worker and per-request embedding_model parameter fit the project's pure-C policy, and no code change is requested. Read the linked #1112 and CONTRIBUTING.md first, then the fork prototype at commit 6d10b2c7 to see the adapter scope. Done means a maintainer answers the two numbered questions and agrees on focused, reviewable increments.

Written with AI assistance (Codex), on behalf of H4M4CHi-ttr.

Request for guidance before an upstream feature PR

I would like guidance on an optional local EmbeddingGemma 2 backend that callers explicitly select for search_graph semantic search. A prototype exists in my fork; I am asking about the integration boundary before proposing an upstream patch.

Related: #1112, including the maintainer's request to characterize a permissively licensed candidate and the independent companion-tool discussion. Those discussions do not establish approval for this adapter or parameter.

What problem does this solve?

CBM's existing semantic search uses bundled Nomic-derived vectors and native ranking. I want to evaluate retrieval from function/class source bodies for questions such as “Where are failed HTTP requests retried?”, including queries written in Japanese.

This is an evaluation goal, not a claim that Gemma improves retrieval quality or resolves the Spanish-language example in #1112. The model is google/embeddinggemma-2, pinned to revision 914f7f89142e33e77833254d9c9b90c3cef7303b; its model card specifies Apache 2.0.

Proposed solution and boundaries

The fork-only prototype is at 6d10b2c7. The provisional selector is embedding_model: "embeddinggemma-2", valid with semantic_query. Omitted or "default" selection preserves existing semantic search. Gemma uses its own index and cosine retrieval; this prototype does not fuse two models' results or scores.

• Inference uses an optional local Python/PyTorch worker. Transformer libraries and weights are not linked into the C binary, but the C adapter, build definition, and MCP schema do change.
• One supervised worker loads on the first explicit Gemma request and is reused within that MCP session. Separate sessions consume separate model memory. The adapter uses argument-vector spawning without a shell and owner-private, bounded JSON files; it adds no listening port.
• The worker is trusted code running with the same OS user's privileges, not an OS sandbox. It opens the native SQLite DB with mode=ro and writes a separate vector DB. Its dependence on the internal native schema needs an agreed integration contract.
• After CBM's native watcher publishes changes, the next Gemma search refreshes an existing compatible index, reusing unchanged document vectors and atomically publishing the replacement DB. Initial indexing and profile/library upgrades remain explicit.
• Before/after source checks reject detected inconsistencies. Worker/CUDA failures are explicit errors, without silently selecting the default model or CPU. An editing agent must still reopen current source before modifying it.

Candidates are verified source-backed Function/Method/Class nodes. The fixed profile is text-only, FP32, 768 dimensions, L2 normalization, and a 4096-token limit including the document title/path; truncations are counted. Freshness checks add corpus-dependent I/O.

Configuration prepares the runtime; tool arguments select the model

Users first prepare a native CBM index, optional Python dependencies, cached weights, and an initial Gemma index for each target project. The first model download is allowed only through explicit indexing with --allow-download.

An MCP client's launch configuration can supply:

CBM_GEMMA2_PYTHON=/absolute/path/to/venv/python
CBM_GEMMA2_SCRIPT=/absolute/path/to/tools/embeddinggemma2.py
CBM_GEMMA2_DEVICE=cuda
CBM_GEMMA2_PRELOAD=0

With preload unset or 0, the first explicit Gemma search loads the model. With 1, cached weights load during MCP initialize, before its response; failure fails initialization. Preloading neither builds a project index nor changes the default model. These environment variables must be supplied before CBM starts.

Actual selection is per request:

{
  "project": "example",
  "semantic_query": ["Where are failed HTTP requests retried?"],
  "embedding_model": "embeddinggemma-2",
  "semantic_limit": 5
}

An agent can receive that instruction in a prompt or its project instructions; a direct MCP client can supply the argument programmatically. Standing instructions should apply only to projects with a prepared Gemma index, honor an explicit user request for the default model, report errors, and leave embedding_model off BM25-only or structural-only calls. Agent instructions encourage parameter use; the actual tool call is the evidence of selection. CBM does not interpret agent prompts/instruction files to choose a model.

The prototype has no environment/configuration switch for changing the default search model. Explicit selection exposes each project's index requirement and the additional inference cost to the caller.

Fresh verification of the feature-only commit

I rebuilt and reran verification on 6d10b2c7, without cherry-picking the separate Windows fix in #2639 and without changing the source tree.

Environment: Windows x64, MSYS2 CLANG64/Clang 23.1.3, Python 3.12.15, GTX 1070 Ti 8 GiB, driver 581.80, PyTorch 2.14.0+cu126/CUDA 12.6, sentence-transformers 6.1.0, transformers 5.19.0.

The one C failure is the unchanged watcher_stop_and_unwatch_cancel_blocked_git_without_backstop fixture: cbm_setenv("PATH", path, 1) returns EINVAL when its old narrow getenv reads the non-ASCII toolchain PATH. The same test binary passed that case with an ASCII process PATH and its ASan DLLs available. The native watcher/application/test files match the fork base; their UTF-8 fix remains separate in #2639. Real CUDA integration used ASCII local repository/cache paths.

A first MCP attempt under Windows' default Temp was refused by CBM's native private-directory check because ancestor ACLs grant mutation to sandbox users. A dedicated user-owned local cache succeeded; no global ACL or MCP settings were changed.

Re-executed source checks:

scripts/build.sh CC=clang CXX=clang++ BUILD_DIR=build/gemma-feature-only-prod-20261014
scripts/test.sh --suites semantic,mcp,watcher,daemon_application CC=clang CXX=clang++ BUILD_DIR=build/gemma-feature-only-tests-20261014
python tests/test_embeddinggemma2.py -v
scripts/security-audit.sh
scripts/check-dco.sh main..review/embeddinggemma-2

This is local integration evidence, not a blind quality benchmark, a full-suite/full-tree-lint pass, cross-platform validation, or established upstream CI coverage. No full-corpus adapter latency/quality claim is made. A matched evaluation still needs fixed repositories, queries and gold labels, declared candidate coverage, strong lexical/structural baselines, cold/warm latency, indexing/refresh costs, and resource measurements. English/Japanese results alone cannot resolve the Spanish use case.

Alternatives and questions

1. Does an optional supervised local worker fit the pure-C and external-service policies, or should it remain an independent companion using existing public CLI/MCP outputs?
2. If an adapter is acceptable in principle, is explicit per-request model selection with lazy loading/optional preload appropriate? The parameter name and schema integration are provisional.

The fork is an experimental reference, not a request to merge its full patch as-is. Any agreed upstream work would need focused, reviewable increments under CONTRIBUTING.md. I am not submitting an upstream Gemma PR with this RFC.

• I searched existing issues; this RFC focuses on the worker/API contract and links the broader capability request in #1112.

The prototype commit carries a human-approved DCO Signed-off-by.