autoInject recall silently drops memory injection on long / non-Latin prompts (HTTP 400 Query too long)

Closed Beginner friendly
  • Is something wrong?
#5,476 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

A pull request for this has already been merged.

  • #5514 by @nicoloboschi — merged

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
typescript
Domain
tooling

Research direction

The bad call is client.recallObservations(prompt.slice(0, 2e3)) inside injectRecall, which the issue says is duplicated across the tsup bundles in dist/. Start in the shared TypeScript source for that call, not the bundles, and check how the error is swallowed. Done when a long non-Latin prompt stays under the 500-token limit and still gets memory injected, and a recall failure logs a warning instead of returning undefined silently.

Written by the indexing model from the issue text.

Description

integration:coding-agents

Summary

When autoInject is set to "recall", the plugin builds the recall query directly from the raw user prompt:

const observations = await client.recallObservations(prompt.slice(0, 2e3), { timeoutMs });

prompt here is the current turn's user message (it can be long). The server rejects any recall query longer than 500 tokens:

HTTP 400  Query too long: N tokens exceeds maximum of 500.

When that happens the exception is swallowed — injectRecall catches and returns undefined, the caller retries once (resolveInjection), then gives up. Because the "recall" mode has no page fallback, the turn ends up with no memory injected at all (silently). The "pages" path is unaffected because it uses prompt.slice(0, 500).

This is easy to hit: the 2000-character slice is a character budget, not a token budget. For CJK text 2000 chars ≈ 1000–1100 tokens; other non-Latin scripts and emoji are even heavier.

Environment

  • @vectorize-io/[email protected]
  • Harness: opencode (harness id opencode2), self-hosted Hindsight server
  • Config: {"serverMode":"self-hosted","apiUrl":"http://localhost:8888","autoInject":"recall","injectTimeoutMs":15000}

Reproduction

  1. Set autoInject to "recall".
  2. Start a session and send a prompt whose first 2000 characters exceed 500 tokens
    (e.g. ~2000 Chinese characters, or ~1300 Russian/Arabic characters).
  3. Observe: no memory is injected for that turn.

Observed diagnostic log (inject_recall_failed), from the plugin's own diag.jsonl:

{"event":"inject_recall_failed","tokens":1011,...}
{"event":"inject_recall_failed","tokens":1061,...}

Observed server response:

HTTP 400  Query too long: 1011 tokens exceeds maximum of 500.

Short prompts work fine, so the failure is silent and intermittent (only on long prompts).

Root cause

A fixed character slice (2e3) is used as a proxy for a token budget against a hard server-side token limit (500). The two only coincide for ASCII-heavy text.

Suggested fix

Bound the query by an estimated token budget with headroom, rather than a fixed character count. Minimal change:

// was: prompt.slice(0, 2e3)
client.recallObservations(prompt.slice(0, 500), { timeoutMs })

A safer, script-aware version (over-estimates cost so it never exceeds the limit):

const RECALL_QUERY_MAX_TOKENS = 400; // 20% headroom under the server's 500

function truncateRecallQuery(s: string, maxTokens = RECALL_QUERY_MAX_TOKENS): string {
  let t = 0, out = "";
  for (const ch of s) {
    const cp = ch.codePointAt(0)!;
    const cost = cp < 0x80 ? 0.3 : cp > 0xffff ? 2 : 1; // ASCII / BMP-non-ASCII / non-BMP
    if (t + cost > maxTokens) break;
    t += cost; out += ch;
  }
  return out;
}

// ...
const observations = await client.recallObservations(truncateRecallQuery(prompt), { timeoutMs });

Ideally the client itself clamps the query to the server's advertised limit (or the server returns the limit in a header) so callers cannot hit this at all.

Additional notes

  • The same prompt.slice(0, 2e3) call is duplicated across all tsup bundles in dist/ (opencode2, claude-hook, codex-hook, cursor-hook, …), so a fix should live in the shared source, not per-bundle.
  • Consider logging a warning (instead of silently returning undefined) when recall fails, so memory loss is observable.
Dominant language
Python
Stars
45.2k
Forks
5.9k
Avg merge
3h 44m
Merged PRs (30d)
491

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from vectorize-io/hindsight

All issues in vectorize-io/hindsight

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.