feat(sandbox-adapter): Kubernetes provider built on Agent Sandbox

The proposed provider would live in packages/sandbox-adapter/src/kubernetes.ts, with docker.test.ts as the model for tests. Read the Docker provider first to see how it runs commands and file operations. Do not start coding yet: the issue asks maintainers to choose between pods/exec, sandboxd and Sandbox Agent, and that decision is still open. Done means the runtime is chosen and the provider's create, connect, suspend and destroy paths are agreed.

Summary

We'd like to contribute a SandboxProvider for Kubernetes, built on Kubernetes Agent Sandbox (SandboxClaim / SandboxTemplate / SandboxWarmPool), at packages/sandbox-adapter/src/kubernetes.ts.

The main thing we need from you is which runtime to implement: how the provider runs commands and file operations inside the sandbox pod. There are three candidates:

1. pods/exec: the Kubernetes exec API, as the Docker provider uses docker exec. Nothing extra in the image.
2. sandboxd: Agent Sandbox's own in-pod runtime daemon.
3. Sandbox Agent: the sandbox-agent server running in the pod.

We recommend pods/exec, so that, like the other providers, the sandbox image needs nothing installed. We'll build whichever you prefer. The claim lifecycle (create, connect, suspend, destroy) is the same for all three. Because this provider depends on cluster-side setup, we'd like to agree on the design before writing the tests and docs.

Motivation

• Teams that run their own Kubernetes clusters (for data residency, cost, or GPU access) have no sandbox-adapter provider today. Docker covers local development only.
• Agent Sandbox is the Kubernetes SIG Apps project for exactly this workload. It reached v1.0 and gives claims, warm pools, per-sandbox Services, managed NetworkPolicies, and suspend/resume. It works with any runtime class, so gVisor (GKE Sandbox) or Kata provide the isolation.
• We're building toward running piDurable() sessions this way, with the Rivet engine and the sandboxes in the same cluster.

Proposed API

import { agentSandboxProvider } from "@rivet-dev/sandbox-adapter/kubernetes";

export const agent = piDurable({
  // ...
  sandbox: agentSandboxProvider({
    namespace: "sandbox",
    warmPool: "coding-agent",        // SandboxWarmPool to claim from
    cwd: "/workspace",
    container: "agent",              // container to exec into; defaults to the first
    readyTimeoutMs: 300_000,         // a cold claim includes node scale-up
    suspend: "pod",                  // "pod" | "none"
    claim: { labels, podLabels, volumeClaimTemplates, shutdownAfterSeconds },
    kubeConfig,                      // default: in-cluster ServiceAccount, else ~/.kube/config
  }),
});

SandboxProvider	Implementation
create	Create a SandboxClaim whose name is derived from the actor id (DNS-1123 safe). A retried create adopts the existing claim instead of failing. Returns the claim name.
connect	Read the claim. Return undefined if it's gone or terminating. If its Sandbox is suspended, set spec.operatingMode: Running. Wait for Ready, then return a Sandbox for the claim's pod.
suspend	Set the Sandbox's spec.operatingMode: Suspended. The pod is removed; the Sandbox and its PVCs are kept.
destroy	Delete the claim. Agent Sandbox removes the Sandbox, pod, Service and PVCs.

The runtime options

Option 1 (recommended): pods/exec

The Sandbox operations run through the Kubernetes exec API (@kubernetes/client-node's Exec), the same way the Docker provider uses docker exec:

• exec runs /bin/sh -c, with the working directory, environment and command passed as positional arguments rather than interpolated into the script. It streams stdout and stderr to onData and reads the exit code from the exec status channel.
• Each command starts in its own process group and records its PID in the sandbox. Abort and timeout kill that group explicitly with a second exec. The provider doesn't rely on a closed exec stream killing the process: with non-TTY exec, Kubernetes can leave it running after the client disconnects. On connect, the provider kills any group left over from a previous actor generation, so a resumed run never shares the workspace with an orphaned command.
• readFile streams cat. writeFile pipes content through stdin. mkdir, stat, readdir and exists use mkdir -p, test and a POSIX-sh directory listing, as the Docker provider does.

The only image requirement is /bin/sh with cat and mkdir. Exec goes through the API server, so every command is authenticated, authorized and audit-logged by Kubernetes. It also works on gVisor and Kata pods, where port-forwarding does not.

Options 2 and 3: sandboxd or Sandbox Agent

Both connect directly to the pod through the per-sandbox Service (status.serviceFQDN), so the API server is off the data path. They require a daemon in the sandbox image.

Comparison

	1. pods/exec (recommended)	2. sandboxd	3. Sandbox Agent
Image requirement	/bin/sh	sandboxd as the container's process	sandbox-agent server as the container's process
Data path	API server → kubelet	Direct to the pod (REST :8080, gRPC :9090)	Direct to the pod (HTTP :2468)
Access control	Kubernetes RBAC (pods/exec), audited	NetworkPolicy only; no authentication in its in-cluster modes	NetworkPolicy plus a bearer token
Command when the connection drops	Keeps running; the provider kills it explicitly	SIGKILLed with its process group (per its user guide)	Keeps running; output is re-read by sequence, so a long command survives a network blip
Client in TypeScript	@kubernetes/client-node	gRPC + REST (Agent Sandbox's TypeScript client isn't published yet)	HTTP + JSON; exec uses runRemoteProcess

• sandboxd is Agent Sandbox's standard runtime (KEP-539.2). It suits clusters that want the direct path and already lock sandbox ingress down with NetworkPolicy.
• Sandbox Agent suits teams already using it in their sandboxes. It's also the most tolerant of long commands over unreliable connections.

Whichever you choose, we'd keep the runtime behind an internal interface, so another one could be added later behind a runtime option without changing the provider's API.

What we've verified

Tested on GKE 1.36 with a GKE Sandbox (gVisor) node pool and Agent Sandbox v1.0.7:

1. Exec works on gVisor sandbox pods through the Kubernetes API (kubectl exec), including running commands and reading files.
2. Suspend and resume. Patching the Sandbox to Suspended removed the pod in about 4 seconds, and the claim reported Ready=False (SandboxSuspended). The claim controller didn't revert the patch. Resuming took about 9 seconds with a node already available. With a PVC from the template's volumeClaimTemplates, the workspace survived. With an emptyDir it was wiped, so suspend needs a persistent workspace volume.
3. Cleanup. Deleting the claim removed the Sandbox, pod, Service and PVC.
4. Cold start. A claim from an empty warm pool, including a node scale-up from zero, was ready in about 2 minutes.
5. Sandbox Agent runtime (0.5.3): processes survive the client disconnecting. Client A started a 40-second process, followed its logs to sequence 6, and disconnected. Client B, from another pod, found it still running 15 seconds later. logs?since=6 returned 7, 8, 9…, all 40 entries were contiguous, and exitCode was reported.

Still to verify before the PR:

• Exec from an in-cluster ServiceAccount with namespace-scoped RBAC.
• Whether a dropped exec connection kills the process under runc and gVisor, which determines how much the explicit kill matters.
• Binary-safe writeFile and large readFile round-trips.

Cluster requirements (would go in the docs)

• Agent Sandbox v1.0+ with the extensions (SandboxClaim, SandboxTemplate, SandboxWarmPool).
• A SandboxTemplate and SandboxWarmPool. For suspend, the template also needs a volumeClaimTemplates entry mounted at the workspace.
• RBAC for the actor's ServiceAccount in the sandbox namespace:
  • sandboxclaims: create, get, list, watch, patch, delete;
  • sandboxes: get, watch, patch;
  • with pods/exec: pods get and pods/exec create.
• With sandboxd or Sandbox Agent: service: true on the template, the daemon in the sandbox image, and a NetworkPolicy (through the template's networkPolicy) that admits the actor's namespace on the daemon's ports.

We'd include example YAML for all of the above. The runtime class (gVisor, Kata or the default) is left to the user.

Tests

• Unit tests against a fake Kubernetes API, plus a fake exec stream or a fake runtime server depending on the chosen runtime, like docker.test.ts.
• An integration test against a kind cluster with Agent Sandbox installed and the default runtime class. Happy to wire it into CI, or keep it opt-in, whichever you prefer.

Questions for maintainers

1. Which runtime should we implement: pods/exec, sandboxd, or Sandbox Agent? We recommend pods/exec, but this is your call, and it's the one answer we need before starting.
2. Should the export be ./kubernetes or ./agent-sandbox?
3. Is @kubernetes/client-node acceptable as an optional peer dependency?
4. suspend on a template without a workspace PVC: should it refuse, warn, or silently delete the pod?
5. Should the kind integration test run in CI?

If the direction looks right, we'll open the PR with the provider, tests, a section in docs/content/docs/sandboxes.mdx, and an example under examples/docs/sandboxes/.