Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

OME: SkillClusterUpdated persists a 1024-dim vector, inflating run_record ~18x

オープン
  • 何か問題がありますか？
#398 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
35/100
issue の種類
リファクタリング
明瞭さ
説明が足りない
活発さ
静か
技術スタック
python
領域
backend, database

調査の方向性

Read _stores/run_record.py, engine.py (around lines 686 and 749), and _background/crash_recovery.py (around line 63) to trace how event payloads are stored and replayed. Compare the audit-projection and re-embedding options, paying particular attention to whether recovery can select a different skill when the vector is unavailable. Done means the recovery-divergence question is resolved and a storage approach is chosen.

索引モデルが issue の本文から書いたものです。

説明

What

SkillClusterUpdated.case_vector (added in #393) carries the triggering case's 1024-dim embedding so extract_agent_skill avoids a second embedding call. The event payload is persisted verbatim, so the vector lands in two stores:

  • run_record.event_payload (_stores/run_record.py, column is TEXT NOT NULL)
  • the APScheduler jobstore, while a job is queued (engine.py:686 serializes the event into args)

Measured with a realistic payload:

with case_vector : 14,074 bytes
without          :    765 bytes   -> 18.4x, +13.3 KB/record

At the default max_records_per_strategy = 1000, the skill_cluster_updated ring buffer is ~14 MB instead of ~0.8 MB. It has no audit value — nobody reads 1024 floats out of a run record.

The vector is only consumed when a cluster holds more skills than MAX_SKILLS_IN_PROMPT (extract_agent_skill.py:265-273), which for a highly-aggregated artifact like a skill should be the minority of runs. Most of the time the 13 KB rides along unused.

Why it wasn't fixed in #393

The obvious fix — strip large fields when persisting to run_record — is not local, and this is the part worth carrying forward:

Crash recovery replays the persisted payload to rebuild the event (_background/crash_recovery.py:63 -> engine.py:749 model_validate_json). A trimmed persisted copy deserializes with case_vector is None, which is a legal value that silently routes the recovered run down the md-ordering fallback branch while the original run took the LanceDB-ranked branch. Same event id, different selection, no error. So trimming needs either a payload/audit split with recovery reading the untrimmed copy, or an explicit "vector unavailable after recovery" signal.

Options

  1. Audit projection. Give BaseEvent a hook (or pass model_dump_json(exclude=...)) used only for the run_record write, keeping the recovery path on the full payload. Fixes the audit bloat, leaves the jobstore copy.
  2. Don't carry the vector; re-embed in the > MAX_SKILLS_IN_PROMPT branch. Costs one embedding call on the minority path. Note this does not reintroduce the eventual-consistency dependency #393 removed: that branch already reads LanceDB (find_topk_relevant_in_cluster) and already degrades to md ordering when the index is stale.
  3. Keep as-is and document the sizing (done in CHANGELOG 1.2.3 + #393 body).

Option 2 is the smaller diff and removes the field entirely; option 1 preserves the saved call. Either needs the recovery-divergence question answered first.

Not urgent

Bounded by the ring buffer, so it does not grow without limit. Filed so ome.db sizing is traceable to a decision rather than an accident.

Found during post-merge-review of #393.

主要言語
Python
スター
13.3k
フォーク
926
平均マージ
17時間 8分
マージ済み PR（30日）
32

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

EverMind-AI/EverOS のほかの issue

EverMind-AI/EverOS の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。