Start the container image from a trained AOT cache

Open
  • Is something wrong?
#987 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
22/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
clickhouse, docker, github-actions, java, spring-boot
Domain
backend, ci-cd, devops, infrastructure, performance

Research direction

Work lives on branch feat/987-aot-cache (no PR yet); training replays src/test/resources/flows against a throwaway ClickHouse during the image build and ships /app/riptide.aot. Read the proposed image/ENTRYPOINT changes, make aot-clickhouse-up, host-network docker build, and the PR image job plus release.yml cosign/provenance impact. Done is the two acceptance scenarios (AOT-linked classes before Starting RiptideApplication, and a failing aot-check if the JVM rejects the cache) plus answers to the open questions on warm-up, QEMU arm64 time, and host-network CI—shipping is deferred until those costs are known.

Written by the indexing model from the issue text.

Description

enhancement

Split out of #924 (Phase 1, "Implement AOT Method Profiling (JEP 515 / Project Leyden)") so #924 can close without it. A working prototype exists on a branch with no PR yet. The decision to ship it is deferred until the costs below are understood.

Problem statement

#924 states that the JVM's warm-up after a restart causes packet drops, and that an AOT cache trained on the daemon would remove that window. Neither the drop window nor the gain has been measured on riptide: edge-01 averages about 0.03 cores, and the prototype's only start-up figure is one local sample. Who is hurt, and how often, is Unknown.

User story

  • As an operator restarting or redeploying riptide while exporters keep sending,
  • I want to riptide to reach full decoding speed sooner after start,
  • so that fewer flows are lost in the first minute after a restart.

Proposed solution

The prototype on branch feat/987-aot-cache trains a JDK 25 AOT cache (JEP 483/514/515) during the image build. It replays the flow captures in src/test/resources/flows through the image's own JVM against a throwaway ClickHouse, ships /app/riptide.aot, and fails the build if the JVM rejects the cache. Container image only. The deb/rpm and Nix paths stay without a cache.

Alternatives considered

  • Train in a separate CI job and layer the cache on: works for any ClickHouse binary, but needs per-arch builds and a manifest merge, and changes the cosign and provenance steps in release.yml.
  • Switch the base image to glibc (eclipse-temurin:25) so ClickHouse runs inside the build: bigger image and a base change operators see.
  • Do nothing: no build-time cost and no image change. The warm-up cost, if there is one, stays.

Acceptance criteria

Scenario: the image uses its cache

  • Given an image built from the branch,
  • When it starts with JDK_JAVA_OPTIONS=-Xlog:aot,
  • Then the log shows Using AOT-linked classes: true before Starting RiptideApplication.

Scenario: a cache the JVM rejects never ships

  • Given a build where the class path changes between training and the end of the build,
  • When the image is built,
  • Then the build fails with aot-check: the JVM did not use the AOT cache and nothing is pushed.

Business impact & priority

  • Value: shorter warm-up after restarts, if a warm-up loss exists (Unknown)
  • Frequency: every container start. edge-01 restarted 2 to 3 times a day in September 2026 (Pyroscope, cause unknown).
  • Target metric: time from process start to /readyz and UDP drops in the first 60 s under load, on the flow-knee lab. Current and target values Unknown.
  • Priority: Unknown

What the prototype showed

Build time:

  • Each image build runs riptide for about 45 s of training per architecture, plus a ClickHouse start. A local multi-arch build took 1 min 40 s.
  • The arm64 half trains under QEMU on a GitHub runner, never measured. publish-rc takes 4 min 31 s to 5 min 36 s today.
  • The build needs a ClickHouse on the build host's network: make aot-clickhouse-up, docker build --network host, and in CI a host-network buildx builder with the network.host entitlement.
  • No PR built the image before. The prototype adds a PR image job (amd64).

Runtime compatibility, all measured on the prototype image:

  • The cache is valid only for the image's JDK, class path and -XX:+UseCompactObjectHeaders.
  • The JVM skips it, with one warning line and no failure, under -XX:-UseCompactObjectHeaders, -XX:+UseZGC, a heap of about 30 GiB or more (-Xmx40g, or MaxRAM=120g with no container memory limit), or any class path change. -Xmx8g, -Xmx512m and -XX:+UseParallelGC keep it.
  • The image runs the extracted Spring Boot layout. /app/riptide.jar is now a thin jar with /app/lib/, and a replaced CMD naming -jar /app/riptide.jar keeps working.
  • -XX:AOTCache sits on the ENTRYPOINT. Replacing the entrypoint drops the cache.
  • The image grows by the cache, 83 MB uncompressed on arm64.

Start-up: one local sample, not a measurement, Spring context start 1.367 s without the cache and 0.469 s with it.

Training excludes the Cisco WLC captures because of #985.

Open questions

  • Is there a warm-up drop window at all at production load? This decides whether the rest matters.
  • How long does arm64 training under QEMU take on a GitHub runner, and is that acceptable for every publish-rc and release?
  • Is a build-time dependency on a host-network ClickHouse and the network.host entitlement acceptable for the release pipeline?
  • Do operators run ZGC, heaps of 30 GiB or more, or unlimited containers on large hosts, where the cache is skipped?
  • What priority does this get?
Dominant language
Java
Stars
2
Forks
0
Avg merge
1d 1h
Merged PRs (30d)
52

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Riptide-Labs/riptide

All issues in Riptide-Labs/riptide

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.