feat(integrations): add NeMo Relay observability sanitization

The integration code lives under integrations/nemo-relay. Start by reviewing draft PR #286 for the current implementation and test coverage. Understand Relay's callback registration and the Anonymizer in-memory API from issue #281. The work involves designing a worker that reuses one Anonymizer pipeline, meets strict latency and memory limits, and ensures no mutation of live traffic. Testing requires proving fail-closed behavior and successful redaction in a real Relay/Codex run.

Priority Level

Medium (Nice to have)

Is your feature request related to a problem?

NeMo Relay copies LLM, tool, scope, and mark payloads into its observability pipeline. Those copies can contain names, contact details, credentials, and other sensitive text before they reach ATOF, OpenTelemetry/Phoenix, or custom subscribers.

Relay has a sanitizer extension point for protecting those copied values, but NeMo Anonymizer does not currently ship a supported Relay integration. Callers would otherwise need to maintain a private adapter, write in-memory values to temporary files, or export the unsanitized data.

The integration must stay off the application path: it must not change provider requests, model responses, tool arguments, or tool results. It also has to fit Relay's 30-second worker callback limit. The current full Anonymizer pipeline does not: an exact-worker qualification measured 116.852 seconds for the cold call and 39.862 seconds for the warm call.

Describe the solution you'd like

Add a first-party NeMo Relay integration under integrations/nemo-relay, maintained with Anonymizer source and packaged as a Relay Python worker.

provider and tool traffic ───────────────────────────► unchanged
        │
        └─ Relay-owned observability copy
                         ↓
                 Anonymizer worker
                         ↓ protected copy
                    Relay runtime
                         ↓
          ATOF / OTLP / Phoenix / custom subscribers

The worker should:

• register Relay's LLM request/response, tool request/response, mark, and scope sanitizer callbacks;
• accept records directly in memory and preserve caller IDs and input order;
• use Relay's active request/response codec where that surface exposes one;
• sanitize the semantic annotation and exact copied provider payload without changing the live request;
• remove recognized credentials, media, and large encoded values before sending text to detector or evaluator models;
• reuse one initialized Anonymizer pipeline rather than rebuilding it per event;
• initialize the pipeline lazily or outside Relay's registration RPC; any prewarm must not make activation exceed Relay's deadline;
• return protected copies to Relay's normal subscriber fan-out;
• rely on Relay's existing fail-closed behavior so mutable observability fields are cleared when sanitization fails or times out; and
• restore secrets and remove temporary artifacts during normal shutdown.

This integration is for observability sanitization only. Request-path policy enforcement, rewriting live application traffic, and offline dataset anonymization are out of scope.

Acceptance criteria

Public Anonymizer contract

• A released Anonymizer version provides typed in-memory text records and stable result correlation, as tracked by #281.
• Existing file-based input and CLI behavior remain compatible.
• The integration pins an exact released Anonymizer version; it does not depend on a source checkout or an open version range.

Relay integration behavior

• LLM request/response, tool request/response, mark, scope-start, and scope-end copies are covered.
• Provider requests, tool calls, and application results remain byte-for-byte outside the sanitizer's mutation boundary.
• Streaming chunk payloads have a documented safe behavior.
• Active Relay codecs are used where available, and returned provider payloads are shape-validated.
• Event identity, timestamps, parentage, names, and categories are documented as immutable fields that must not carry PII.
• Sanitizer failure and timeout tests prove fail-closed omission before subscriber fan-out.

Runtime and lifecycle

• The worker reuses one Anonymizer pipeline, and any prewarm strategy keeps activation within Relay's registration deadline.
• A supported online profile stays comfortably below Relay's 30-second callback deadline under representative agent traffic.
• Normal completion, callback failure, cancellation, and shutdown release worker-owned resources.
• Temporary data is removed even if Relay forcibly terminates a worker with an in-flight synchronous model call, or all temporary data lives under a host-owned directory that Relay removes.
• The packaged environment remains below Relay's 512 MiB limit on every supported platform with reasonable headroom.
• Worker configuration validates endpoints, model names, secret environment mappings, text budgets, cache bounds, and provider timeouts.

Qualification and release

• Unit and contract tests cover projection, codec round trips, caching, lifecycle, timeout, and non-mutation.
• CI builds the integration on Relay's supported Python/platform matrix.
• A checksummed release archive installs, validates, and activates through Relay's normal plugin flow.
• A real Relay/Codex run proves that seeded PII is redacted in ATOF and Phoenix while benign payload content remains available.
• Documentation clearly states the supported surfaces, probabilistic false-positive/false-negative limits, model data boundary, latency budget, and fail-closed behavior.

Current evidence

Draft PR #286 contains the current implementation and test coverage.

The exact worker has already demonstrated:

• bundle installation, validation, and activation;
• one Anonymizer instance reused across multiple calls;
• cleanup after completed calls;
• a real Codex task completing normally through Relay;
• zero seeded PII reaching ATOF or Phoenix when callbacks timed out; and
• trace identity and relationships remaining visible after payload omission.

That run did not prove successful live redaction. The full pipeline exceeded Relay's deadline, so Relay correctly cleared the mutable payloads.

A separate fresh-process constructor-only run used unreachable detector and evaluator endpoints with health checks disabled. Construction succeeded without contacting either endpoint, but took 313.244 seconds on the same macOS environment. Draft PR #286 therefore keeps initialization lazy: awaiting construction inside worker registration would exceed Relay's 30-second limit before the plugin became active. A detector-only experiment was fast enough (about 0.13 seconds warm and 3.03 seconds for 64 mixed records), but over-redacted benign coding strings such as GPU-A100, pytest -q, and Ru. A fast profile therefore needs an explicit label policy and its own quality qualification rather than being silently enabled.

The current macOS arm64 Python 3.11 environment measured 475.780 MiB, leaving 36.220 MiB under Relay's 512 MiB limit.

Dependencies and related work

• #281 provides the required in-memory API.
• #280 tracks a possible detector-only mode; adopting it is a separate quality and product decision.
• Draft implementation: #286.

Describe alternatives you've considered

• Write temporary input files before every call: adds avoidable I/O and weakens cleanup for a request-oriented worker.
• Sanitize on the live provider/tool path: changes the safety boundary and adds Anonymizer latency to application execution.
• Run a private exporter after Relay: duplicates ATOF/OTLP/Phoenix delivery and can allow unsanitized parallel subscribers.
• Use the current full pipeline unchanged: it exceeds Relay's callback deadline even after the first call.
• Silently switch to detector-only mode: meets the latency budget in early testing but currently produces unacceptable false positives on ordinary coding text.

Additional context

The intended ownership model is the same one used by other upstream-maintained Relay integrations: Anonymizer owns the source and its public API contract, while Relay owns worker execution, callback deadlines, copied-event fan-out, and plugin activation. No benchmark harnesses, credentials, Phoenix databases, screenshots, or generated traces should be committed to the repository.