Spike: FIPS-compliant agent runtimes

Start with build-cli-linux.yml and the pinned goose 1.51.0 submodule, then rerun the dependency inventory against goose main. Get written answers on the FIPS boundary and validated-versus-system crypto, run check-payload against the project images in CI, and assess the upstream, carried-patch, and SDK routes. Done means a costed recommendation, an early CI check, follow-up issues, and an ADR if needed.

Context

The MTA 8.3 release workflow runs an openshift/check-payload
FIPS scan that does not run against stage builds. It scanned 20 related images
and failed four:

The Go findings have an answer downstream —
migtools/mta-agentic-controller#20
is merged. The equivalent change here, #251, is closed unmerged, so this
repository still builds its Go binaries without the FIPS settings the product
build now has. Whether to revive #251 is part of this spike.

Goose was not in the failure list, and that is not a pass we earned. The
check-payload build ART ran predates its Rust support, so the goose binary
was never actually examined. When a newer scanner reaches it, it fails.

We prototyped a fix for that ahead of the release
(migtools/mta-agentic-controller#19,
closed). It built clean on both arches and produced a goose that defines no
crypto symbols of its own and links the system libcrypto. It was closed
because it was not needed for 8.3 and it solved the wrong problem: it satisfied
what the scanner inspects, which is not the same as being FIPS-compliant.

This spike is about the real question, for all of our agent runtimes rather than
just the one that got scanned.

What we already know about goose

Resolved from cargo tree -p goose-cli --edges normal --target x86_64-unknown-linux-gnu against the pinned 1.51.0 submodule, using the default
features that build-cli-linux.yml ships for Linux. Runtime edges only, so
build-script dependencies are excluded. Recorded here so the spike does not
start from zero:

• The binary contains two independent bundled crypto libraries.
  aws-lc-rs/aws-lc-sys (from the goose dependency itself, jsonwebtoken,
  rcgen, and rustls) and ring.
• ring arrives by two separate paths, neither obvious. First,
  rustls-tls includes "sqlx/tls-rustls", and sqlx 0.9 defines
  tls-rustls = ["tls-rustls-ring"]. Second, the otel feature reaches
  opentelemetry-otlp → opentelemetry-http → reqwest 0.13 →
  rustls-platform-verifier → rustls-webpki with its default ring backend.
  That reqwest is a different major version from goose's own, so feature
  unification does not reach it. Fixing either path alone leaves ring in
  place — verified.
• This contradicts upstream's own intent.
  aaif-goose/goose#10116,
  authored by a Red Hat engineer, gated rcgen/aws_lc_rs behind rustls-tls
  specifically to keep the backends apart.
• More crypto sits outside TLS entirely. sigstore-crypto (the update
  feature, verifying goose's own release signatures), secp256k1 (nostr
  nip44), and md-5/aes/cbc pulled in by document parsing — lopdf for
  PDF and umya-spreadsheet for XLSX.
• The default build uses no system crypto at all. There is no openssl in
  the graph.

The practical conclusion: no change to goose's default build reaches FIPS
compliance. secp256k1 is not an approved curve, and sigstore-verify has no
system-crypto backend. Anything workable is a separate build profile.

Open questions

• Which definition of compliance binds us. rustls already offers
  fips = ["aws_lc_rs", "aws-lc-rs?/fips"], which builds against AWS-LC-FIPS —
  a genuinely FIPS 140-3 validated module. That is far cheaper than routing
  through OpenSSL. But it is a vendored validated module, and Red Hat product
  policy generally requires the OS-certified module. Get a definitive answer
  from product security rather than inferring one; the two routes differ by
  weeks of work.
• What is actually inside our FIPS boundary. Does an agent runtime whose
  security-relevant crypto is "TLS to a model endpoint" need its own validated
  crypto for PDF decryption and content hashing, or only for the transport?
  A narrow, defensible boundary could shrink this problem substantially. This
  question is worth answering before any code is written.
• Carry, upstream, or replace. Three routes, and we should cost all three:
  carry patches against the goose submodule indefinitely; land a FIPS build
  profile upstream in aaif-goose/goose; or stop shipping goose and build the
  runtime ourselves on an agent SDK, where we control the dependency graph from
  the start. The third was raised half-seriously during the 8.3 scramble and
  deserves a real estimate rather than a joke.
• Upstream appetite. Is aaif-goose/goose willing to take a FIPS profile?
  There is an existing Red Hat contributor with a merged series in exactly this
  area, and upstream CI already runs a native-tls matrix job, so the ground is
  more favourable than it looks. No FIPS issue has been filed there yet.
• The skills container is a different bug. /etc/redhat-release missing is
  a base-image question, not a crypto one, and needs its own answer.

Timeboxed tasks

• Re-run the dependency inventory against goose main. The numbers above
  come from our 1.51.0 pin and upstream has drifted.
• Get a written answer on vendored-validated versus system-certified, and
  on where our FIPS boundary is drawn.
• Run check-payload against our own images in CI, so a release workflow
  is not the first thing to tell us. This is likely the single highest-value
  item here regardless of what else we decide.
• File the narrow upstream issue about the two crypto backends — it stands
  on its own merits as a bug against #10116's intent — and gauge the
  response before proposing anything larger.
• Decide whether #251 should be revived, and reconcile it with the approach
  that merged downstream. The two PRs took different routes.
• Estimate the "own runtime on an SDK" option honestly enough to compare.

Deliverables

• A recommendation naming one route, with a cost attached to each of the three.
• A CI check that catches this class of failure before a release workflow does.
• Follow-up issues for whatever the recommendation implies, and an ADR if the
  decision changes what we ship.

Out of scope

• The Go binaries themselves, beyond deciding the fate of #251.
• Operator and bundle packaging.
• FIPS certification as a process. This is about what we build, not about who
  signs off on it.