[Routing]: activate routing in the api proxy server

Summary

The controller from issue #8941 does not know where its inputs come from or who reads its answer, and the running proxy never calls it. This issue connects both ends. They land together because a session with no server to start it is unreachable, and a server wired to a missing session cannot be reviewed.

What this adds, and why

• routing-runtime.js binds the controller to the real proxy and to the private files the host watches. It loads the conversation under a one MiB bound, publishes the selection atomically, and still publishes a failure record when something breaks after the agent has started.
• The server starts routing only after key validation and model discovery succeed, so a run never routes against a catalogue it has not confirmed.
• Admitted routed requests use the enforcement transform and routed upgrades are rejected, so the selection is enforced from the first agent request.
• Shutdown drains in-flight work before the servers close, so a completion record never appears for a run that was still streaming.
• The Dockerfile copies the twelve routing runtime modules, so the published image contains what the server now loads.

What stays the same

A deployment without an apiProxy.routing block behaves exactly as it does today. One with the block fails closed, rejecting every inference request, until issue 06 supplies the private mounts and the container conversation path.

Background

These issues add task-level model routing to the firewall. Instead of naming a
model, a workflow says what it wants from the run, and the firewall picks one
model and one reasoning effort for the whole run. The router itself is the
separate githubnext/gh-aw-router
service, which the firewall runs as a credential-free sidecar. It ranks the
candidates the API proxy offers it, the proxy executes that choice, and the host
hands it to the agent before the agent starts.

The motivation is cost and durability. A workflow pinned to a single model
overpays for routine work and underperforms on hard work, and it breaks outright
when that model is deprecated, renamed, or withdrawn. Routing chooses from the
live model catalogue under the allow and deny policy and budget guards that
already exist, so as long as the catalogue still offers an eligible model the
workflow keeps running as that catalogue changes and spends against a stated
objective rather than a hardcoded model name.

Routing is opt-in and fails closed. Without configuration nothing changes, and a
routing failure aborts the run instead of quietly falling back, so a routed
workflow never runs on a model that nobody chose. The router is given no
provider credential, so it can rank models but never call one, and it runs only
from a digest-pinned image.

One limitation is known and unresolved. Routing works only when the configured
provider is Copilot, because Copilot is the only provider whose model catalogue
publishes reasoning efforts, supported endpoints, and context limits. The others
report little more than a model id and when it was observed, and the candidate
builder skips any model missing efforts or protocol. Pointing routing at OpenAI
or Anthropic therefore produces an empty pool and a clean no_route failure
rather than a bad route. Closing this needs a capability-description method on
the provider adapter plus a per-provider mapping into it, which would improve
what awf --reflect reports at the same time. That work is deliberately
outside this series.

The work lands as numbered issues in order. The early issues add contracts and
leaf modules that nothing imports yet, the middle issues assemble them inside
the proxy, and the last issues wire the host and prove the result end to end.
Every issue is expected to leave the repository green on its own.

Dependencies

This change depends on no open issue in this sequence, so apply it on current main.

What is already on main

Every issue above this one has merged by the time this change is applied. Build
on what they added rather than adding a second version of it.

From issue #8851, the routing configuration surface and the wire contracts, merged as #8853.

• The typed apiProxy.routing block, parsed through both the config file and the build-config path.
• router as a recognized role in the container.images manifest.
• The canonical JSON schema, its generated runtime copy, and the configuration reference section.
• Closed AJV 2020 schemas in routing-contract.js for every routing payload, plus the one-MiB planning bound.
• Bounded failure codes and sanitized failure records in routing-errors.js. toRoutingFailure trusts only a real RoutingError.
• Strict opt-in parsing of AWF_ROUTING_CONFIG in routing-config.js, which returns null when the variable is absent.
• The recorded router corpus and the corpusCase accessor the contract tests replay.
• ajv pinned at 8.20.0 in the proxy package file and lockfile.

From issue #8872, the routable model candidate pool, merged as #8891.

• One provider-aware policy matcher shared by the HTTP request guard and the model resolver, with denylist precedence unchanged.
• A resolver whose auto pass-through also checks that policy, so a denied auto or copilot/auto cannot slip past the resolver.
• A resolver whose middle-power fallback picks only from policy-permitted models, so an unavailable request cannot resolve to a denied one.
• routing-catalogue.js, which turns live Copilot metadata into routable records carrying authoritative efforts, protocols, and capacity.
• routing-candidates.js, which returns a frozen, deterministically ordered pool numbered from choice-0001 with a null-prototype lookup back to the native model name.
• A pool that offers a model and effort pair only on the endpoint enforcement will later require for it.

From issue #8917, the router client, the trusted request context, and the metered classifier path, merged as #8918.

• routing-router-client.js, the only planner HTTP client, bounded by response size, attempt time, cancellation, and an overall deadline, sending no credential, and fixing each endpoint method so a caller cannot override it.
• routing-planner.js, which retries a transport failure up to three attempts, including EPIPE and EAI_AGAIN, treats a contract failure as terminal, and keeps the last transport error as the cause of router_unavailable. ENOTFOUND is deliberately terminal and reaches the caller as the raw error after one attempt.
• A trusted in-process request context that no header and no request body can set.
• Suppression of steering, private diagnostics, and cache-miss counting for a classification request, with token and AI credit accounting still active and recording the purpose. A request carrying req.awfRouting also skips steering.
• routing-classifier.js, which builds one deterministic non-streaming request in the native protocol of the candidate and refuses a candidate that cannot hold the reply.
• routing-provider-executor.js, which sends it back through proxyRequest so credentials, guards, and accounting keep one implementation.

From issue #8941, the routing decision controller and its enforcement, merged as #8942.

• routing-controller.js, which produces either one immutable selection or one sanitized failure while reading no environment variable and writing no file.
• The degradation rules for capacity exclusions, invalid classifier answers, and terminal provider failures.
• The selection log fields, including the degradation reason, attempt count, eligible choices, catalogue overlap, and latency.
• routing-enforcement.js, which screens every agent inference request against the one selection and rejects a mismatch rather than rewriting it. It matches the parsed pathname, so a query string does not change which endpoint is enforced.

How the router differs from other sidecars

Reviewers reasonably ask why the router gets handling the other sidecars do not.

What is new, and why

• Failure and completion records are new. No other sidecar makes a claim about the run that the host has to check afterward. Routing claims the whole run used one model, so the proxy records a failure that happens after the agent starts, and exits 78 when it cannot, rather than let the host report a success nobody verified.

Acceptance criteria

• A conversation is read under an explicit one MiB bound, and filesystem errors are translated at the public boundary rather than inside the low-level reader.
• The session refuses to plan when a result file already exists, so a stale selection is never reused.
• The selection is published atomically, and a failure after selection publishes a runtime failure record.
• A routing failure that cannot be published terminates the proxy with exit code 78 rather than leaving the host waiting.
• Routing starts only after key validation and model discovery have succeeded.
• An admitted routed request uses the enforcement body transform rather than the adapter transform, and a routed upgrade is rejected before the normal tunnel handler.
• Shutdown drains in-flight work before the servers close, and completion is published only after log and telemetry shutdown.
• The published image contains all twelve routing runtime modules and no fixtures or tests.
• Without AWF_ROUTING_CONFIG the proxy behaves exactly as it does today.

Out of scope

Do not change host staging or configuration, and do not enable routed WebSocket requests.

Notes on the design

These record why the change is shaped the way it is. Read them before adapting
anything below, because several of them rule out a change that would otherwise
look like a reasonable simplification.

• Copy the complete factory signatures and their defaults, not just the wiring expression. A missing raw configuration returns null without activating routing.
• loadRoutingConversation calls readPrivateRoutingJson(filePath, 1_048_576). The helper default of 16_384 bytes is for result records, not conversations. The low-level reader preserves filesystem errors such as ENOENT for its callers to translate.
• A failure after selection publishes runtime-failure.json. If that publication fails, terminate the proxy with 78. Never allow a logging or failure callback to throw into the response stream.
• shutdown aborts and drains. completeShutdown is a separate final step and cannot publish completion before draining.
• Screen inference after health and reflect but before revealing adapter configuration.
• Main already forwards a configured apiProxy.routing block as AWF_ROUTING_CONFIG, added by pull request 8876, with the host conversation path and no private mounts. From this issue on, such a configuration therefore fails closed, because routing cannot load the conversation and every inference request is rejected, until issue 06 supplies the mounts and the container path. Keep that failure. Do not default the missing values.

Implementation

Everything from here down is a recommendation drawn from a working reference
branch, not a patch to apply. Each file is named, with the code it gains quoted
exactly as the reference has it. New files are shown whole unless the section
says otherwise, and an existing file shows only what it gains. Adapt the
surrounding context where current main requires it, but preserve the public
APIs, payloads, ordering, error behavior, and assertions shown here.

The routing session

containers/api-proxy/routing-runtime.js

New file. It binds the controller to the real proxy dependencies and to the private files the host watches, covering bounded conversation loading, atomic publication of the selection, and the failure records that must still appear after the agent container has started.

/**
 * Production wiring for private model routing in the AWF API proxy.
 *
 * Assembles the routing controller from environment configuration and the live
 * Copilot adapter, then exposes it as a session that startup.js starts once and
 * drains on shutdown.
 *
 * Session lifecycle:
 *   start()            → plan → write selection.json
 *   enforcement        → admit only the selected model for the agent's request
 *   shutdown()         → abort in-flight work, drain, then write complete.json
 *
 * The input and output directories belong to the proxy alone and are the only
 * channel back to the host, so every file there is opened with O_NOFOLLOW and
 * checked for size and link count before it is parsed.
 */

'use strict';

const privateFs = require('fs');
const path = require('path');
const { randomUUID } = require('crypto');
const { TextDecoder } = require('util');
const { normalizePolicyList } = require('./routing-candidates');
const { createRoutingCatalogue } = require('./routing-catalogue');
const { parseRoutingConfig } = require('./routing-config');
const { createRoutingController } = require('./routing-controller');
const { createRoutingError, RoutingError, toRoutingFailure } = require('./routing-errors');
const { createRoutingEnforcement } = require('./routing-enforcement');
const { createRoutingProviderExecutor } = require('./routing-provider-executor');
const { createRoutingRouterClient } = require('./routing-router-client');
const { cachedModels } = require('./key-validation');
const { logRequest } = require('./logging');
const { checkRateLimit, proxyRequest } = require('./proxy-request');
const { getRuntimeModels } = require('./runtime-model-catalog');

/**
 * Read the staged conversation the planner classifies.
 *
 * @param {string} filePath Path inside the private input directory.
 * @param {{ signal?: AbortSignal }} [options]
 * @returns {Promise<unknown>} Parsed conversation.
 * @throws {RoutingError} Always a routing error, so parse details never surface.
 */
async function loadRoutingConversation(filePath, { signal } = {}) {
  if (signal?.aborted) {
    throw createRoutingError('routing_cancelled', 'Model routing was cancelled');
  }
  try {
    return readPrivateRoutingJson(filePath, 1_048_576);
  } catch (error) {
    if (error instanceof SyntaxError || error instanceof TypeError || error instanceof RoutingError) {
      throw createRoutingError('routing_contract_error', 'The private routing conversation is invalid');
    }
    throw createRoutingError('routing_configuration_error', 'The private routing conversation is unavailable');
  }
}

/**
 * Route routing stage records into the proxy log at the matching severity.
 *
 * @param {(level: string, event: string, record: object) => void} [writeLog]
 * @returns {{ record: (record: object) => void }}
 */
function createRoutingObserver(writeLog = logRequest) {
  return Object.freeze({
    record(record) {
      writeLog(record.stage === 'failure' ? 'warn' : 'info', 'model_routing', record);
    },
  });
}

/**
 * Parse an allow/deny model policy supplied as a raw JSON environment value.
 *
 * @param {string|undefined} raw
 * @param {string} name Environment variable name, used in the error message.
 * @returns {string[]|null}
 */
function parsePolicyList(raw, name) {
  let value;
  try {
    value = raw === undefined ? null : JSON.parse(raw);
  } catch {
    throw createRoutingError('routing_configuration_error', `${name} must contain valid JSON`);
  }
  return normalizePolicyList(value, name);
}

/**
 * Compose a routing controller from environment configuration.
 *
 * Returns null when routing is not configured. The controller is dormant: it
 * contacts no router until run() is called.
 *
 * @returns {{ run: Function }|null}
 */
function createProductionRoutingController({
  rawConfig = process.env.AWF_ROUTING_CONFIG,
  getCopilotAdapter,
  routerTransport,
  observer = createRoutingObserver(),
  clock,
  random,
} = {}) {
  const config = parseRoutingConfig(rawConfig);
  if (config === null) return null;
  const policy = {
    allowedModels: parsePolicyList(process.env.AWF_ALLOWED_MODELS, 'AWF_ALLOWED_MODELS'),
    disallowedModels: parsePolicyList(process.env.AWF_DISALLOWED_MODELS, 'AWF_DISALLOWED_MODELS'),
  };
  if (typeof getCopilotAdapter !== 'function') {
    throw createRoutingError('routing_configuration_error', 'The Copilot provider adapter owner is unavailable');
  }

  const catalogue = createRoutingCatalogue({
    getCopilotAdapter,
    getDiscoveredModels: provider => cachedModels[provider],
    getRuntimeModels,
  });
  const executor = createRoutingProviderExecutor({
    getCopilotAdapter,
    proxyRequest,
    checkRateLimit,
  });

  return createRoutingController({
    config,
    planner: createRoutingRouterClient({ transport: routerTransport }),
    catalogue,
    policy,
    loadConversation: loadRoutingConversation,
    executor,
    observer,
    routerIdentity: Object.freeze({ name: 'gh-aw-router' }),
    ...(clock ? { clock } : {}),
    ...(random ? { random } : {}),
  });
}

/**
 * Read and parse a JSON file from the private routing directories.
 *
 * Validates the opened descriptor rather than the path, so the file cannot be
 * swapped for a link or grown between the check and the read.
 *
 * @param {string} filename
 * @param {number} [maxBytes]
 * @returns {unknown}
 */
function readPrivateRoutingJson(filename, maxBytes = 16_384) {
  const descriptor = privateFs.openSync(filename,
    privateFs.constants.O_RDONLY | privateFs.constants.O_NOFOLLOW | privateFs.constants.O_NONBLOCK);
  try {
    const stat = privateFs.fstatSync(descriptor);
    if (!stat.isFile() || stat.nlink !== 1 || stat.size > maxBytes) {
      throw createRoutingError('routing_contract_error', 'Invalid private routing file');
    }
    const buffer = Buffer.alloc(maxBytes + 1);
    let length = 0;
    while (length < buffer.length) {
      const count = privateFs.readSync(descriptor, buffer, length, buffer.length - length, null);
      if (count === 0) break;
      length += count;
    }
    if (length > maxBytes) throw createRoutingError('routing_contract_error', 'Oversized private routing file');
    return JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(buffer.subarray(0, length)));
  } finally {
    privateFs.closeSync(descriptor);
  }
}

/**
 * Write a routing result the host will read, atomically and exactly once.
 *
 * The host treats the first result it sees as authoritative, so an existing
 * file is an error rather than something to overwrite.
 *
 * @param {string} outputDir
 * @param {string} name
 * @param {object} record
 */
function publishRoutingResult(outputDir, name, record) {
  const content = JSON.stringify(record);
  if (Buffer.byteLength(content) > 16_384) {
    throw createRoutingError('routing_contract_error', 'Routing result exceeds its byte limit');
  }
  const filename = path.join(outputDir, name);
  const temporary = path.join(outputDir, `.${randomUUID()}.tmp`);
  try {
    if (privateFs.existsSync(filename)) throw createRoutingError('routing_contract_error', 'Routing result already exists');
    privateFs.writeFileSync(temporary, content, { flag: 'wx', mode: 0o600 });
    privateFs.renameSync(temporary, filename);
  } finally {
    privateFs.rmSync(temporary, { force: true });
  }
}

/**
 * Build the routing session the proxy owns for the lifetime of the run.
 *
 * Returns null when routing is not configured. The returned object also spreads
 * the enforcement surface (screenRequest, rejectUpgrade, drain) so callers hold
 * a single routing handle.
 *
 * @returns {object|null}
 */
function createProductionRoutingSession({
  rawConfig = process.env.AWF_ROUTING_CONFIG,
  getCopilotAdapter,
  outputDir = '/run/awf-routing/output',
  createController = createProductionRoutingController,
  observer = createRoutingObserver(),
  fatalExit = code => process.exit(code),
} = {}) {
  if (rawConfig === undefined) return null;
  const abortController = new AbortController();
  const deadline = Date.now() + 90_000;
  let runPromise;
  let result;
  let terminalFailure;
  let drained = false;

  // Terminal failure reported from anywhere in the run. Called from the live
  // agent response path, so it must never throw back into that stream.
  function recordFailure(code) {
    if (terminalFailure || (result && !result.ok)) return;
    terminalFailure = toRoutingFailure(createRoutingError(code, 'Routed execution was rejected'));
    try {
      observer.record({ stage: 'failure', phase: result?.ok ? 'primary' : 'bootstrap', code: terminalFailure.code });
    } catch {}
    if (!result?.ok) {
      abortController.abort();
      return;
    }
    try {
      publishRoutingResult(outputDir, 'runtime-failure.json', terminalFailure);
    } catch {
      fatalExit(78);
    }
  }

  const getSelection = () => result?.ok ? result.selection : null;
  const getFailure = () => terminalFailure || (result && !result.ok ? result.failure : null);
  const enforcement = createRoutingEnforcement({ getSelection, getFailure, recordFailure });

  async function execute() {
    try {
      for (const name of ['selection.json', 'failure.json', 'runtime-failure.json', 'complete.json']) {
        if (privateFs.existsSync(path.join(outputDir, name))) {
          throw createRoutingError('routing_contract_error', 'Stale routing results cannot be reused');
        }
      }
      const controller = createController({ rawConfig, getCopilotAdapter, observer });
      result = await controller.run({ signal: abortController.signal, jobDeadlineMs: deadline });
      if (terminalFailure) result = Object.freeze({ ok: false, failure: terminalFailure });
      publishRoutingResult(outputDir, result.ok ? 'selection.json' : 'failure.json',
        result.ok ? result.selection : result.failure);
    } catch (error) {
      result = Object.freeze({ ok: false, failure: toRoutingFailure(error instanceof RoutingError ? error :
        createRoutingError('routing_configuration_error', 'Private routing bootstrap failed')) });
      if (!privateFs.existsSync(path.join(outputDir, 'selection.json')) &&
          !privateFs.existsSync(path.join(outputDir, 'failure.json'))) {
        publishRoutingResult(outputDir, 'failure.json', result.failure);
      }
    }
    return result;
  }

  return Object.freeze({
    start() {
      if (!runPromise) runPromise = execute();
      return runPromise;
    },
    async shutdown() {
      abortController.abort();
      await runPromise;
      await enforcement.drain();
      drained = true;
    },
    completeShutdown() {
      if (!drained) throw createRoutingError('routing_contract_error', 'Routing shutdown is incomplete');
      if (result?.ok) publishRoutingResult(outputDir, 'complete.json', { schema: 'awf-routing-complete/v1' });
    },
    getSelection,
    getFailure,
    ...enforcement,
  });
}

module.exports = {
  createProductionRoutingController,
  createRoutingObserver,
  loadRoutingConversation,
  createProductionRoutingSession,
  readPrivateRoutingJson,
};

Turning it on in the server

containers/api-proxy/server-factory.js

Existing file. It screens inference after the health and reflect handlers, uses the enforcement body transform for admitted routed requests, and rejects routed upgrades.

4 places change, in file order.

function createProxyHandler(adapter, checkRateLimit, proxyRequest) {
  return (req, res) => {
    const contentLength = parseInt(req.headers['content-length'] || '0', 10);
    if (checkRateLimit(req, res, adapter.name, contentLength)) return;

    if (adapter.transformRequestUrl) {
      req.url = adapter.transformRequestUrl(req.url);
    }

    // A routed request carries its own transform, which pins the selected model.
    // Using the adapter's transform here would reintroduce aliases and fallback.
    const bodyTransform = req.awfRouting ? req.awfRouting.bodyTransform : adapter.getBodyTransform();

    proxyRequest(
      req, res,
      adapter.getTargetHost(req),
      adapter.getAuthHeaders(req),
      adapter.name,
      adapter.getBasePath(req),
      bodyTransform,
      adapter.getRequestSigner ? adapter.getRequestSigner() : null,
      adapter.getTargetScheme ? adapter.getTargetScheme(req) : 'https'
    );
  };
}

Inside createProviderServer:

    proxyRequest,
    proxyWebSocket,
    routing,
  } = deps;

    }

    // Runs before the enabled check so a rejected request never reveals provider configuration.
    // Returns true once it has written a 403. Otherwise it may attach
    // req.awfRouting, which pins the body to the selected model downstream.
    if (routing?.screenRequest(req, res, adapter)) return;

    if (!adapter.isEnabled()) {
      const response = adapter.getUnconfiguredResponse

  });

  server.on('upgrade', (req, socket, head) => {
    // A routed run pins one model, which an opaque tunnel would bypass.
    if (routing) {
      routing.rejectUpgrade(socket);
      return;
    }
    handleUpgrade(req, socket, head);
  });
  server.shutdownConnections = () => handleUpgrade.shutdownConnections();

containers/api-proxy/server.js

Existing file. It drains in-flight routed work before the servers close and publishes completion last.

4 places change, in file order.

const { logRequest } = require('./logging');
const {
  MODEL_ALIASES,
  MODEL_FALLBACK,
  parseModelFallbackConfig,
  makeModelBodyTransform: makeModelBodyTransformForProvider,
  filterResolvableAliases,
  filterAvailableModelsToConfiguredProviders,
  getEffectiveModelFallbackForReflect,
} = require('./model-config');
const {
  keyValidationResults,
  cachedModels,
  getRuntimeCatalogSnapshot,
  configureKeyValidation,
  resetKeyValidationState,
  resetModelCacheState,
  isKeyValidationComplete,
  isModelFetchComplete,
  setKeyValidationComplete,
  setModelFetchComplete,
  refreshProviderModelsForResolution,
  probeProvider,
  validateApiKeys,
  fetchStartupModels,
  validateRequestedModel,
} = require('./key-validation');
const { createProviderServer: createProviderServerFactory } = require('./server-factory');
const { bootPrimary } = require('./startup');
const { createProductionRoutingSession } = require('./routing-runtime');

const registeredAdapters = createAllAdapters(process.env, {
  openaiBodyTransform: makeModelBodyTransform('openai'),
  anthropicBodyTransform: makeModelBodyTransform('anthropic'),
  copilotBodyTransform: makeModelBodyTransform('copilot'),
  geminiBodyTransform: makeModelBodyTransform('gemini'),
});
const routing = createProductionRoutingSession({
  getCopilotAdapter: () => registeredAdapters.find(adapter => adapter.name === 'copilot'),
});

function createProviderServer(adapter) {
  return createProviderServerFactory(adapter, {
    handleManagementEndpoint,
    reflectEndpoints,
    checkRateLimit,
    proxyRequest,
    proxyWebSocket,
    routing,
  });
}

if (require.main === module) {
  bootPrimary({
    registeredAdapters,
    createProviderServer,
    validateApiKeys,
    fetchStartupModels,
    writeModelsJson,
    validateRequestedModel,
    setKeyValidationComplete,
    setModelFetchComplete,
    closeLogStream,
    otelShutdown,
    logRequest,
    HTTPS_PROXY,
    routing,
  });
}

containers/api-proxy/startup.js

Existing file. It starts routing only after key validation and model discovery have succeeded.

5 places change, in file order.

Inside bootPrimary:

  logRequest,
  HTTPS_PROXY,
  routing,
}) {
  logRequest('info', 'startup', {

      Promise.all(oidcInitPromises).then(() => {
        const validation = validateApiKeys(adaptersToStart).catch((err) => {
          logRequest('error', 'key_validation_error', { message: 'Unexpected error during key validation', error: String(err) });
          setKeyValidationComplete(true);

          setModelFetchComplete(true);
          writeModelsJson();
        }).then(async () => {
          // Routing needs a validated key to reach /models and the fetched
          // catalogue to build candidates, so it waits on both.
          if (routing) {
            await validation;
            await routing.start();
          }
        }).catch(() => {
          logRequest('error', 'model_routing_bootstrap_error', { message: 'Private model routing bootstrap failed' });
        });
      });

    }, Number.isFinite(forceExitMs) && forceExitMs > 0 ? forceExitMs : 8000);
    forceExitTimer.unref();
    await routing?.shutdown().catch(() => {
      logRequest('error', 'model_routing_shutdown_error', { message: 'Model routing shutdown failed' });
    });
    await Promise.all(startedServers.map((server) => {
      if (typeof server.shutdownConnections === 'function') {

    await otelShutdown();
    clearTimeout(forceExitTimer);
    try {
      routing?.completeShutdown?.();
    } catch {
      // 78 is the routing-failure code the CLI reports to the caller.
      process.exit(78);
      return;
    }
    process.exit(0);
  }

containers/api-proxy/Dockerfile

Existing file. It copies the twelve routing runtime modules into the published image, so the image contains what the server now loads, and copies no fixtures or tests.

# Copy application files
COPY server.js logging.js metrics.js rate-limiter.js rate-limiter-window.js \
     token-tracker.js token-persistence.js token-parsers.js \
     token-tracker-http.js token-tracker-ws.js token-tracker-shared.js \
     model-resolver.js model-fallback.js model-utils.js model-body-rewriter.js proxy-utils.js oidc-adapter-utils.js adapter-factory.js anthropic-transforms.js claude-hosted-web.js codex-hosted-web.js hosted-web-policy.js \
     model-config.js key-validation.js server-factory.js startup.js \
     proxy-request.js request-headers.js upstream-http.js proxy-guards.js proxy-error-handler.js http-client.js body-handler.js model-discovery.js management.js oidc-token-provider.js \
     oidc-token-provider-base.js \
     github-oidc.js aws-oidc-token-provider.js aws-sigv4.js gcp-oidc-token-provider.js \
     anthropic-oidc-token-provider.js \
     ai-credits-pricing.js models-dev-catalog.js models.dev.catalog.json \
     provider-pricing-overlays.js runtime-model-catalog.js \
     routing-candidates.js routing-catalogue.js \
     routing-classifier.js routing-config.js routing-contract.js routing-controller.js \
     routing-errors.js routing-enforcement.js routing-planner.js \
     routing-provider-executor.js routing-router-client.js routing-runtime.js \
     oidc-refresh-utils.js body-transform.js body-utils.js codex-compat.js rate-limit.js websocket-proxy.js \
     websocket-guards.js websocket-tunnel.js \
     deprecated-header-tracker.js billing-headers.js upstream-response.js \
     upstream-log.js upstream-retry.js upstream-token.js \
     anthropic-cache.js otel.js otel-exporters.js otel-serialization.js otel-workload-identity.js \
     token-budget-log.js blocked-request-diagnostics.js \
     provider-env-constants.js provider-env-constants.json provider-names.js \
     model-api-mapping.js model-api-mapping.json ./
COPY guards/ ./guards/
COPY providers/ ./providers/
COPY transforms/ ./transforms/

Tests

This change adds 18 cases. They belong in the same pull request as the
code they cover. A suite listed by case name is quoted only by its titles,
because the titles state what each case must prove and the bodies follow from
the implementation above.
A few suites also quote the cases that are hardest to reinvent, such as
ordering, idempotence, and failure isolation, together with the setup they
share.

containers/api-proxy/routing-runtime.test.js

New file. It pins the one MiB conversation bound, the watch-before-check ordering, the exit code 78 path, and the separation of draining from publishing completion.

The 15 cases this change adds:

• does nothing when routing configuration is absent
• composes a dormant controller without contacting the router
• accepts valid raw model policy %s
• marks only native GitHub Copilot adapters as routing providers
• loads private conversation JSON without exposing its path in failures
• logs only the controller record at the production observer boundary
• publishes one private selection
• publishes controller failure without selection
• preserves selection and the first runtime rejection independently
• does not publish selection after an early inference rejection
• cancels an in-flight run and publishes a closed failure
• publishes completion only after a successful drain
• never reruns classification over stale %s
• refuses %s private routing data
• does not create a session without opt-in configuration

The setup these cases share, then the cases that pin the behavior hardest to get right, quoted in full:

describe('private routing session', () => {
  let directory;
  let outputDir;
  let session;
  const selection = {
    schema: 'awf-routing-selection/v1',
    engine: 'copilot',
    provider: 'copilot',
    choice: { id: 'choice-0001', model: 'github-copilot/gpt-5-mini', effort: 'high' },
    wire_model: 'gpt-5-mini',
  };

  beforeEach(async () => {
    directory = await fs.mkdtemp(path.join(os.tmpdir(), 'awf-routing-session-'));
    outputDir = path.join(directory, 'output');
    await fs.mkdir(outputDir, { mode: 0o700 });
  });

  afterEach(async () => {
    await session?.shutdown();
    session = undefined;
    await fs.rm(directory, { recursive: true, force: true });
  });

  // Build a session backed by a stub controller that returns `result`, or runs
  // `plan` when a case needs control over when planning settles.
  function createSession(result = { ok: true, selection }, plan = async () => result) {
    const run = jest.fn(plan);
    const createController = jest.fn(() => ({ run }));
    session = createProductionRoutingSession({ rawConfig, outputDir, createController });
    return { createController, run };
  }

  it('publishes one private selection', async () => {
    const { createController, run } = createSession();
    const pending = session.start();
    expect(session.start()).toBe(pending);
    await expect(pending).resolves.toEqual({ ok: true, selection });
    expect(readPrivateRoutingJson(path.join(outputDir, 'selection.json'))).toEqual(selection);
    expect((await fs.stat(path.join(outputDir, 'selection.json'))).mode & 0o777).toBe(0o600);
    expect(await fs.readdir(outputDir)).toEqual(['selection.json']);
    expect(createController).toHaveBeenCalledTimes(1);
    expect(createController).toHaveBeenCalledWith(expect.objectContaining({ rawConfig }));
    expect(run).toHaveBeenCalledTimes(1);
    expect(session.getSelection()).toEqual(selection);
  });

  it('preserves selection and the first runtime rejection independently', async () => {
    createSession();
    await session.start();
    const response = { writeHead: jest.fn(), end: jest.fn() };
    const request = { method: 'POST', url: '/session', headers: {} };
    expect(session.screenRequest(request, response, { name: 'copilot' })).toBe(true);
    const firstFailure = readPrivateRoutingJson(path.join(outputDir, 'runtime-failure.json'));
    expect(firstFailure).toMatchObject({
      schema: 'awf-routing-failure/v1', code: 'model_routing_mismatch', retryable: false,
    });
    session.rejectUpgrade({ write: jest.fn(), destroy: jest.fn() });
    // A second rejection must not overwrite the first recorded reason.
    expect(readPrivateRoutingJson(path.join(outputDir, 'runtime-failure.json'))).toEqual(firstFailure);
    expect(readPrivateRoutingJson(path.join(outputDir, 'selection.json'))).toEqual(selection);
    expect(session.getSelection()).toEqual(selection);
    expect(session.getFailure()).toEqual(firstFailure);
  });

  it('publishes completion only after a successful drain', async () => {
    createSession();
    await session.start();
    expect(() => session.completeShutdown()).toThrow('incomplete');
    await session.shutdown();
    session.completeShutdown();
    expect(readPrivateRoutingJson(path.join(outputDir, 'complete.json'))).toEqual({ schema: 'awf-routing-complete/v1' });
    expect((await fs.stat(path.join(outputDir, 'complete.json'))).mode & 0o777).toBe(0o600);
  });

  it.each(['selection.json', 'failure.json', 'runtime-failure.json', 'complete.json'])('never reruns classification over stale %s', async name => {
    await fs.writeFile(path.join(outputDir, name), '{}');
    const { createController } = createSession();
    await expect(session.start()).resolves.toMatchObject({ ok: false, failure: { code: 'routing_contract_error' } });
    expect(createController).not.toHaveBeenCalled();
    expect(await fs.readFile(path.join(outputDir, name), 'utf8')).toBe('{}');
  });

containers/api-proxy/server-factory.test.js

Existing file. It pins the handler order, the upgrade rejection, and which body transform is selected.

The 2 cases this change adds:

• uses the routed transform without invoking model aliases or fallback
• rejects routed upgrades before authenticating or opening upstream sockets

The cases that pin the behavior hardest to get right, quoted in full:

  test('uses the routed transform without invoking model aliases or fallback', () => {
    const selection = { choice: { effort: 'high' }, wire_model: 'gpt-5-mini' };
    const adapter = {
      name: 'copilot',
      isEnabled: () => true,
      getTargetHost: () => 'api.githubcopilot.com',
      getAuthHeaders: () => ({}),
      getBasePath: () => '',
      getBodyTransform: jest.fn(),
    };
    const routing = createRoutingEnforcement({
      getSelection: () => selection,
      getFailure: () => null,
      recordFailure: jest.fn(),
    });
    const proxyRequest = jest.fn();
    const server = createProviderServer(adapter, { routing, checkRateLimit: () => false, proxyRequest });
    const request = { url: '/responses', method: 'POST', headers: {} };
    const response = Object.assign(new EventEmitter(), {
      statusCode: 200,
      write: jest.fn(),
      end: jest.fn(),
    });

    server.emit('request', request, response);

    expect(adapter.getBodyTransform).not.toHaveBeenCalled();
    // Argument 6 is the body transform in the proxyRequest signature.
    expect(proxyRequest.mock.calls[0][6]).toBe(request.awfRouting.bodyTransform);
    expect(() => request.awfRouting.bodyTransform(Buffer.from('{"model":"auto"}'))).toThrow('task model selection');
  });

  test('rejects routed upgrades before authenticating or opening upstream sockets', () => {
    const routing = { rejectUpgrade: jest.fn() };
    const adapter = { isEnabled: jest.fn() };
    const proxyWebSocket = jest.fn();
    const server = createProviderServer(adapter, { routing, proxyWebSocket });
    const socket = makeTrackedSocket();

    server.emit('upgrade', {}, socket, Buffer.alloc(0));

    expect(routing.rejectUpgrade).toHaveBeenCalledWith(socket);
    expect(adapter.isEnabled).not.toHaveBeenCalled();
    expect(proxyWebSocket).not.toHaveBeenCalled();
  });

containers/api-proxy/startup.test.js

Existing file. It pins that ordering and proves an unconfigured proxy starts exactly as it does today.

The case this change adds:

• starts routing once after discovery settles, discovery failure=%s

The case that pins the behavior hardest to get right, quoted in full:

  test.each([false, true])('starts routing once after discovery settles, discovery failure=%s', async failed => {
    let settleDiscovery;
    const discovery = new Promise((resolve, reject) => { settleDiscovery = failed ? reject : resolve; });
    const routing = {
      start: jest.fn().mockResolvedValue(undefined), shutdown: jest.fn().mockResolvedValue(undefined),
      completeShutdown: jest.fn(() => expect(server.close).toHaveBeenCalled()),
    };
    const server = { listen: (_port, _host, ready) => ready(), close: jest.fn(done => done()) };
    bootPrimary({
      registeredAdapters: [{
        name: 'copilot', port: 10002, participatesInValidation: true,
        isEnabled: () => true, getTargetHost: () => 'api.githubcopilot.com',
      }],
      createProviderServer: () => server,
      validateApiKeys: jest.fn().mockResolvedValue(undefined),
      fetchStartupModels: () => discovery,
      writeModelsJson: jest.fn(), validateRequestedModel: jest.fn(),
      setKeyValidationComplete: jest.fn(), setModelFetchComplete: jest.fn(),
      closeLogStream: jest.fn(), otelShutdown: jest.fn(), logRequest: jest.fn(), routing,
    });
    await new Promise(resolve => setImmediate(resolve));
    expect(routing.start).not.toHaveBeenCalled();
    settleDiscovery(failed ? new Error('discovery unavailable') : undefined);
    await new Promise(resolve => setImmediate(resolve));
    expect(routing.start).toHaveBeenCalledTimes(1);
    await handlers.SIGTERM();
    expect(routing.shutdown).toHaveBeenCalledTimes(1);
  });

Validation

Use Linux and the repository-supported Node version. Run the unit suites as a
non-root user whose UID is not 1000 because ownership tests target 1000.
Install the root and both proxy dependency sets before importing new modules.

npm ci
npm --prefix containers/api-proxy ci
npm --prefix containers/cli-proxy ci
npm --prefix containers/api-proxy test -- --runInBand --runTestsByPath routing-runtime.test.js server-factory.test.js startup.test.js

Then run the complete existing gates. Do not weaken a test to accommodate the
implementation or count a skipped required suite as a pass.

npm run type-check
npm run lint
npm run build
npx jest --runInBand
npm --prefix containers/api-proxy test -- --runInBand
npm --prefix containers/cli-proxy test -- --runInBand

In the agent PR, report the executed commands and results, confirm the listed
scope, and explain any adaptation required by changes to main since this patch.
Resolve code-review findings and require repository CI to pass before closing
the issue. The previous reference suite totals are not a substitute for this run.

Image smoke check

Run this on the Linux Docker host with routing unset. It builds this checkout,
checks the live health endpoint, and removes only its own test container.

(
set -euo pipefail
image=awf-routing-proxy-smoke:candidate
container="awf-routing-proxy-smoke-$BASHPID"
docker build --tag "$image" --file containers/api-proxy/Dockerfile containers/api-proxy
trap 'docker rm -f "$container" >/dev/null 2>&1 || true' EXIT
docker run --detach --rm --name "$container" "$image"
healthy=false
for attempt in $(seq 1 30); do
  if docker exec "$container" node -e 'const http = require("node:http"); const request = http.get("http://127.0.0.1:10000/health", response => { response.resume(); process.exitCode = response.statusCode === 200 ? 0 : 1; }); request.setTimeout(1000, () => request.destroy()); request.on("error", () => { process.exitCode = 1; });'; then
    healthy=true
    break
  fi
done
test "$healthy" = true
)