Model-call cache barely replays on a re-run of the same head, and identical input yields disjoint findings

オープン
#30 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
typescript

調査の方向性

No files or tests are named. Start by tracing the model-call cache key and the run.json call and token totals, then reproduce the cold and warm runs described in the report. Done means identifying why identical packets miss the cache and making cache hit and miss counts available to callers.

索引モデルが issue の本文から書いたものです。

説明

Problem

Re-reviewing the same head should be close to free: the diff is identical, the packets are identical, and only the reviewer's prior findings differ. In practice a warm run costs almost as much as a cold one.

A/B on 0.5.6, same repository, same head commit, same provider and model, both invocations with --cache, run back to back:

run model calls total tokens
cold 96 2.46M
warm 80 2.17M

That is 17% fewer calls and 12% fewer tokens - within the run-to-run noise of this pipeline rather than a replay.

The second signal is more interesting than the cost: the two runs produced disjoint findings on identical input. Same head, same model, same configuration, no overlap in what came back.

Questions
  1. What exactly is the model-call cache keyed on? If the packet prompt embeds anything that varies per run - ordering of retrieved context, timestamps, a run id, a shuffled symbol list - the key changes on every invocation and the cache can never hit on the work that matters.
  2. Are cache hits and misses counted anywhere a caller can read them? run.json reports call and token totals, but nothing I could find states how many calls were served from the cache. Without that counter, a user cannot tell a cold pipeline from a broken cache - which is exactly the position this report is written from.
Why it matters

Review rounds repeat. A change is pushed, the reviewer runs again, and most packets are byte-identical to the previous round. If those replay, the marginal cost of an extra round approaches zero and re-reviewing becomes routine. If they do not, every round is charged at full price and teams ration the tool.

The disjoint-findings result also makes caching a correctness question, not only a cost one: if identical input can produce non-overlapping output, then a cache hit and a cache miss are not interchangeable, and it matters which one a user got.

Offer

I can re-run the A/B on 0.5.7 with any additional telemetry you want emitted, and report per-packet hit/miss if the counters are exposed.

主要言語
TypeScript
スター
8
フォーク
1
平均マージ
5分
マージ済み PR(30日)
2

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

0xPolygon/codegenie のほかの issue

0xPolygon/codegenie の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。