anthropic: serialize non-strict prompt-injected schema as compact JSON instead of indent=2
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 83/100
Research direction
Read hindsight_api/engine/providers/anthropic_llm.py around AnthropicLLM.call and compare its schema serialization with gemini_llm.py. The issue identifies the non-strict schema injection and proposes compact JSON serialization; done means the injected schema keeps its keys, values, and field order while avoiding pretty-print whitespace.
Written by the indexing model from the issue text.
Description
Bug Description
On the non-strict structured-output path (strict_schema=False, the default for retain), AnthropicLLM.call (hindsight_api/engine/providers/anthropic_llm.py:384) injects the response JSON schema into the system block using json.dumps(schema, indent=2, ensure_ascii=False).
Pretty-printing the extraction schema with indent=2 adds several kilobytes of pure whitespace and newlines to every call (gemini_llm.py:953 already serializes without indent=2). Behind corporate LLM gateways/proxies with strict request-body limits (e.g., 20 KB), even a small retain chunk crosses the payload limit because of the indented schema.
Related Issues Reported by Others
Several recent issues have pointed out the token, cache, and payload overhead of soft-path prompt-injected schemas:
- https://github.com/vectorize-io/hindsight/issues/6187 —
AnthropicLLMnon-strictschema_msginjection in_cached_system_blocks(fixed in https://github.com/vectorize-io/hindsight/pull/6216). - https://github.com/vectorize-io/hindsight/issues/6422 — soft-path
json.dumps(schema, indent=2, ensure_ascii=False)injection inflating prompt tokens and re-billing on every turn. - https://github.com/vectorize-io/hindsight/issues/6349 — soft-schema prompt assembly issues.
- https://github.com/vectorize-io/hindsight/issues/1002 & https://github.com/vectorize-io/hindsight/issues/2504 — overhead of text-injected schemas when
strict_schemais off or unsupported.
Proposed Fix
Use compact JSON serialization (separators=(",", ":")) when building schema_msg in AnthropicLLM.call. Keys, values, and field order remain identical while keeping a typical retain extraction request comfortably under 20 KB without truncating text or metadata.
git diff against v1.2.0 (34a138684)
diff --git a/hindsight-api-slim/hindsight_api/engine/providers/anthropic_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/anthropic_llm.py
index 8110c9135..4f3a11e2b 100644
--- a/hindsight-api-slim/hindsight_api/engine/providers/anthropic_llm.py
+++ b/hindsight-api-slim/hindsight_api/engine/providers/anthropic_llm.py
@@ -381,7 +381,10 @@ class AnthropicLLM(LLMInterface):
schema = _for_native_output(strict_json_schema(response_format))
else:
schema = provider_json_schema(response_format)
- schema_msg = f"You must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2, ensure_ascii=False)}"
+ schema_msg = (
+ "\n\nYou must respond with valid JSON matching this schema:\n"
+ f"{json.dumps(schema, separators=(',', ':'), ensure_ascii=False)}"
+ )
Version
v1.2.0 (34a138684)
- Dominant language
- Python
- Stars
- 45.2k
- Forks
- 5.9k
- Avg merge
- 3h 44m
- Merged PRs (30d)
- 491
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from vectorize-io/hindsight
-
core
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
vectorize-io/hindsight#6430 ·
Maintainers usually reply within 1 day
-
core
Difficulty 1/5 Under an hour Newbie friendliness 75/100
vectorize-io/hindsight#6416 ·
Maintainers usually reply within 1 day
-
core
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
vectorize-io/hindsight#6415 ·
Maintainers usually reply within 1 day
-
core
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
vectorize-io/hindsight#6397 ·
Maintainers usually reply within 1 day
-
core
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
vectorize-io/hindsight#6396 ·
Maintainers usually reply within 1 day
All issues in vectorize-io/hindsight
Similar issues
-
ChatBedrockConverse changes the keys of JSON tool results to camelCase (`order_id` becomes `orderId`)Possibly taken @geyazheng claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
langchain-ai/langchain-aws#1469 ·
Maintainers usually reply within 1 day
-
GCToggler leaves collection enabled if the body re-enables itPossibly taken @Gambit-Checkmate claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 7 days
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
doronz88/pymobiledevice3#2176 ·
Maintainers usually reply within 1 day
-
b4b usability
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day