Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Context caching (ContextCacheConfig) is a silent no-op: config is plumbed to InvocationContext but never read

Open
  • Is something wrong?
#1,530 2 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@hemasekhar-p is already working on this.

Since Sep 23, 2026.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
java
Domain
ai

Research direction

Start by tracing ContextCacheConfig.java through App.java, Runner.java, and InvocationContext.java, then run the supplied recording-BaseLlm reproduction to confirm the request is unchanged. Compare the relevant adk-python cache manager and flow files before deciding the Java package structure. Done requires either working cache creation and request metadata with tests, or an explicit loud failure and corrected support documentation, depending on maintainer direction.

Written by the indexing model from the issue text.

Description

needs review
Summary

ContextCacheConfig is accepted by App.Builder and plumbed all the way to InvocationContext, but nothing in the library ever reads it. Setting it is a silent no-op: no cached content is created, no cachedContent is set on the request, and the full system instruction is sent as tokens on every call.

The context caching docs list Java as supported ("Python v1.15.0+, Java v0.1.0+, Kotlin v0.7.0+"), and RFC #543 was closed with "this feature was successfully implemented in version 0.6.0". I believe that closure was made in error: the commit it points to (12defeed, PR #822) added only the configuration class, and its own message says so — "introduces context caching configuration for apps" (+59 lines in ContextCacheConfig.java, +19/-2 in App.java). PR #823 then added the InvocationContext plumbing. No implementation PR followed.

Because the config is accepted, documented as supported, and plumbed to the point of use, there is no signal that it does nothing — you only find out from a cache-hit rate that never moves.

The chain, traced on main
  1. App.Builder.contextCacheConfig(cfg) stores it on App.
  2. Runner.Builder.build() reads app.contextCacheConfig() (Runner.java:176) and assigns the Runner field.
  3. Runner.newInvocationContextBuilder() passes it to InvocationContext.Builder.contextCacheConfig(...).
  4. InvocationContext.Builder(InvocationContext) copies it, so child contexts carry it.
  5. Reads of that field: equals/hashCode, and the public getter InvocationContext.contextCacheConfig() (InvocationContext.java:303).
  6. That getter has no callers. The trail ends there.

Supporting checks on main:

  • Zero occurrences of CreateCachedContentConfig, CachedContent, caches.create or CacheMetadata anywhere in the repo.
  • ContextCacheConfig.getTtlString() — javadoc'd "Returns TTL as string format for cache creation", i.e. written specifically for this feature — has no callers either.
  • The only cache-adjacent code reads usageMetadata.cachedContentTokenCount back off responses for reporting (GeminiUtil, BigQueryAgentAnalyticsPlugin), which is a different thing.
  • Verified identically against the published 1.9.0 and 1.10.1 sources jars, so this is not new.
Reproduction

Configure caching on an App, run one turn against a recording BaseLlm, and inspect the request:

ContextCacheConfig caching =
    new ContextCacheConfig(/* maxInvocations= */ 10, Duration.ofHours(1), /* minTokens= */ 0);

App app = App.builder()
    .name("repro")
    .rootAgent(LlmAgent.builder().name("repro").model(recordingLlm)
        .instruction("...32KB of static instruction...").build())
    .contextCacheConfig(caching)
    .build();

Runner runner = Runner.builder()
    .app(app)
    .sessionService(new InMemorySessionService())
    .artifactService(new InMemoryArtifactService())
    .build();

// run one turn, then on the LlmRequest the model received:
assertThat(request.config().get().cachedContent()).isEmpty();                    // passes
assertThat(request.config().get().systemInstruction()).isPresent();              // passes — full text, inline
assertThat(app.contextCacheConfig()).isEqualTo(caching);                         // passes — config was kept

Observed, with minTokens = 0 and a ~32KB instruction so no size floor applies:

Prompt supplied as cachedContent on the request System instruction
Instruction.Provider empty inline, full text
static .instruction(String) empty inline, full text
systemInstruction on generateContentConfig empty inline, full text

The request is byte-identical with the config present and absent. Response-side too: a response reporting no cachedContentTokenCount reaches the event with none invented for it, and one reporting cachedContentTokenCount passes through untouched (BaseLlmFlow copies usageMetadata verbatim) — so the reporting channel works and the absence above is genuine.

Impact

For an agent with a large static instruction this is the whole cost saving silently not happening. In our case the evaluator prompt is ~7k tokens, about two thirds of every request's input tokens. Implicit caching does not substitute for it at low request rates — entries live a few minutes, so at single-digit requests per minute spread over several prompt variants we measured roughly a 1% hit rate.

Suggested resolution

Either would resolve the confusion; the first is obviously preferable:

  1. Implement it, porting gemini_context_cache_manager.py, cache_metadata.py and flows/llm_flows/context/_cache.py from adk-python. I'm happy to attempt this as a PR if that would be welcome, but given the size (~1k lines of Python, with non-trivial fingerprint/invalidation state) and that this is a design-sensitive area, I'd rather check first whether it's already planned internally or whether there's a preferred shape — the package-structure question from #543 was never settled publicly.
  2. Failing that, make the no-op loud — reopen #543, correct the docs' Java support claim, and either javadoc ContextCacheConfig as not-yet-implemented on Java or log a warning when an App is built with one. That costs little and stops the next person spending a day proving the negative.

Happy to supply the full repro as a test case if useful.

Environment
  • adk-java main, plus published 1.9.0 and 1.10.1 (behaviour identical)
  • Java 25, google-genai 1.58.0
Dominant language
Java
Stars
1.7k
Forks
431
Avg merge
3d 2h
Merged PRs (30d)
46

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from google/adk-java

All issues in google/adk-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.