Pi: add session-affinity headers and sanitized prompt-cache diagnostics
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
Research direction
No implementation files, tests, or entry points are named in the issue. Start by locating Pi's production request construction and any existing diagnostic tooling, then trace saved-state handling; done means affinity behavior is enabled safely, opt-in diagnostics cover the listed retention modes without exposing bearer tokens, malformed state fails cleanly, and the measured evidence is preserved without asserting a retention guarantee.
Written by the indexing model from the issue text.
Description
Problem
Pi sessions can be routed across multiple AI Gateway destinations when traffic splitting is configured. Prompt caches are destination-local, so a conversation needs session-affinity headers to avoid avoidable cache misses.
Cache-retention investigations also need reproducible, sanitized tooling. Current observations do not prove that GPT subagents have short retention: GPT-5.6 Sol rejects explicit in_memory, and observed expiry may be caused by UAG forwarding, backend eviction, or prefix matching.
Expected behavior
- Enable Pi's production-safe session-affinity compatibility behavior.
- Add opt-in diagnostic scripts that send stable prompt-cache prefixes and report cache usage without printing bearer tokens.
- Cover
24h, provider-default, and explicitin_memoryrequest construction. - Validate phase-separated saved state and fail cleanly on malformed state.
- Preserve the measured 45/50/60-minute evidence without claiming a backend retention guarantee.
- Do not force
PI_CACHE_RETENTIONor install a child/subagent short-cache policy.
- Dominant language
- Python
- Stars
- 97
- Forks
- 75
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 205
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from databricks/unity-gateway
-
feat(codex): support amazon_bedrock Model Provider Services that expose OpenAI-compatible models Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
databricks/unity-gateway#454 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
databricks/unity-gateway#391 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
databricks/unity-gateway#342 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
databricks/unity-gateway#286 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
databricks/unity-gateway#141 ·
All issues in databricks/unity-gateway
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
-
hcocena Openpolicies-accepted pre-review precheck-passed
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Bioconductor/BiocContributions#214 · 5 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
TencentCloud/Octop#1169 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
521xueweihan/HelloGitHub#3778 ·
-
The version checker's trailing attribute region has no control for a less-than inside a quoted value Openarea: dashboard area: tests bug perceived difficulty: 2 python
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Nitjsefnie-Harness-Commons/daedalus#1105 · 1 comment ·