[FEATURE] Socair directory attestation (socair.modeldir/v1) for multi-file Models, verified at download and prefetch

Open
#2,137 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
go, kubernetes
Domain
backend, security

Research direction

Start with internal/controller/model_attestation.go and the existing attestation tests, then review the Model CRD rules and prefetch flow described in the issue. The change also depends on defilantech/socair-verify#1 and relates to #1968 and #2136. Done means multi-file models can be staged and verified against a Socair directory attestation, with mismatches and trust failures reported before the model becomes Ready.

Written by the indexing model from the issue text.

Description

kind/feature priority/low

Feature Description

Socair attestation for multi-file Models: let a Model staged with spec.files (a Hugging Face repo, a sharded safetensors checkpoint) pass the --model-attestation gate using Socair's model directory attestation. Verify the staged files against it wherever the model is downloaded, including the prefetch Job.

Problem Statement

Attestation (#1968) is single-artifact only, so any multi-file Model is refused under enforce and reported unattested under warn:

  • The check needs spec.sha256. internal/controller/model_attestation.go:137: the attestation policy requires spec.sha256, the digest the attestation must be for. The envelope is admitted with policy.Admit([]byte(envelope), model.Spec.SHA256) (line 180).
  • spec.sha256 can't be combined with spec.files. The Model CRD's CEL rule: sha256 verifies a single artifact: it cannot be combined with files or mmproj … (use fileSha256 for multi-file staging).
  • spec.fileSha256 (per-file digests) exists, but the attestation check never reads it.

Multi-file is how large models are served: for example source: nvidia/GLM-5.3-Flash-NVFP4 with files: [config.json, "*.json", "*.jinja", "*.safetensors"], which expands to 44 files (33 shards, about 190 GiB).

Socair already attests these as one artifact. socair scan <dir> attests a model directory. Its subject is the digest of a canonical manifest (socair.modeldir/v1\n, then <sha256> <size> <path>\n for each file, sorted by path), and the report lists every file with its hash (socair docs/dev.md). It's a single subject, so socair-verify's one-subject rule holds. What LLMKube lacks is a way to recompute that digest from the files it stages and admit against it.

Proposed Solution

Admit Socair directory attestations for multi-file Models:

apiVersion: inference.llmkube.dev/v1alpha1
kind: Model
spec:
  source: nvidia/GLM-5.3-Flash-NVFP4
  files: [config.json, "*.json", "*.jinja", "*.safetensors"]
  attestation:
    configMapKeyRef:
      name: glm-5-3-flash-attestation
      key: attestation.dsse.json   # from `socair scan <dir>` + `socair sign`
  • The digest admitted is the staged set's socair.modeldir/v1 manifest digest: every staged file, meaning the concrete spec.files entries plus every glob expansion (status.stagedFiles), each with its SHA-256 and size. It's computed with the helper proposed in defilantech/socair-verify#1 (ModelDirDigest), not re-implemented in LLMKube, so the format can't drift.
  • Where the per-file hashes come from, two modes:
    • Known up front (spec.fileSha256 covers every staged file, or a later digest source): compute the manifest digest and run checkAttestation before any download, as today. Under enforce, an unattested Model never starts a prefetch Job.
    • Not known up front (globs, no fileSha256): check the envelope's signature and trust before downloading. The downloader (the prefetch Job, or the serving init containers) records each file's SHA-256 and size while writing it. The Model only becomes Ready once the recomputed manifest digest equals the attested subject. On a mismatch, att.CompareFiles(staged) (also proposed in socair-verify#1) names the file added, removed or changed. This mirrors Socair's own airlock promote, which renames a directory into place only when its recomputed manifest digest is the attested subject.
  • Prefetch is the natural place for the second mode: the files are hashed away from the GPU nodes, and the InferenceService already waits for Ready (and, with #2136's modelCache.prefetch, holds its workload until the prefetch completes).
  • The attested set must equal the staged set. The attestation has to be made over exactly the files LLMKube stages, so spec.files and the directory scanned with socair scan have to agree. A repo file outside spec.files (a README, original/ weights) that was in the scanned directory makes the digests differ. The condition should say so and name the files, using the attestation's file list.
  • Conditions name the failing file(s) on AttestationVerified / Degraded.
  • Single-file Models are unchanged: configMapKeyRef + spec.sha256 as today.

Derive the staging plan from the attestation (filesFrom: Attestation). A Socair report already describes exactly what to stage, signed. Its artifact carries repo_url, commit_sha, and files[] with path, sha256, size_bytes and role (weights, config, tokenizer, chat_template, code, adapter, other), per socair's report schema v1. So the Model can point at the attestation and drop the hand-written file list:

apiVersion: inference.llmkube.dev/v1alpha1
kind: Model
spec:
  source: nvidia/GLM-5.3-Flash-NVFP4      # must match the attestation's repo_url
  attestation:
    configMapKeyRef:
      name: glm-5-3-flash-attestation
      key: attestation.dsse.json
  filesFrom: Attestation                   # stage exactly the attested files
  • Verify first, then trust the list. The envelope must pass the gate (trusted signature, promotion policy, maxAge) before its file list is used, so filesFrom: Attestation requires a trust policy (--model-attestation-trusted-keys) even in warn mode. An unsigned or untrusted list is never staged. Under off with no keys, the field is rejected with a condition.
  • Stage exactly the attested set: files[].path at commit_sha (the source pinned to that revision), each verified against its attested sha256 and size_bytes while downloading. spec.files and spec.fileSha256 are mutually exclusive with filesFrom: Attestation, enforced in CEL. status.stagedFiles records the attested paths.
  • Check the source: spec.source must name the attestation's repo_url (normalised: org/name, hf://org/name and https://huggingface.co/org/name are equivalent), or the Model is refused. A mirror endpoint (HF_ENDPOINT) doesn't change the repo identity.
  • Optional role policy, for example spec.attestationPolicy.allowRoles or allowCodeFiles: false, refusing to stage role: code files (Python a loader would import) unless allowed, so trust_remote_code surfaces as a decision at admission.
  • What it solves:
    • no hand-maintained file list or globs: whoever ran socair scan / socair airlock pull --file … decided what gets served;
    • the staged set equals the attested set by construction, so the scanned-set-versus-spec.files mismatch above can't happen;
    • every hash is known up front, so the gate runs before any download and prefetch gets a concrete file list (#2136's globs problem);
    • the revision can't drift to a re-pushed branch.
  • Needs from socair-verify: Attestation.Files with Role, plus RepoURL and CommitSHA on Attestation (defilantech/socair-verify#1).

Tests, fed real socair scan <dir> + socair sign output as the existing attestation suite does:

  • a multi-file Model whose staged set matches the attested directory goes Ready under enforce;
  • a staged file with different bytes, or an extra or missing file, refuses the Model and names it;
  • with fileSha256 covering every file, no prefetch Job starts for an unattested Model;
  • filesFrom: Attestation stages exactly the attested files at commit_sha and records them in status.stagedFiles; a spec.source that doesn't match repo_url, an untrusted envelope, or no trust policy refuses the Model before any download; a role: code file is refused when the role policy disallows it;
  • without it, the prefetch Job's recorded hashes drive the digest comparison, and the Model stays out of Ready until it matches.

Alternatives Considered

  • One attestation per file (each single-subject, verified against fileSha256): works with socair-verify v0.1.0 as is, but means one envelope per file (44 here) and doesn't match how Socair attests repositories.
  • Re-implement socair.modeldir/v1 in LLMKube: works today, but the format then lives in two codebases and a subtle difference refuses correctly attested models. socair-verify#1 avoids that.
  • Turn attestation off for multi-file Models: leaves the largest models outside the gate.

Additional Context

  • Depends on defilantech/socair-verify#1: export ModelDirDigest, add Attestation.Files (with roles), RepoURL, CommitSHA and CompareFiles.
  • Related: #1968 (Socair attestation gate) and #2136 (GitOps-safe prefetch), whose prefetch Job would hash and verify the staged files.
  • Docs: docs/security/model-attestation.md here; socair docs/dev.md (directory manifest), docs/airlock.md, docs/kubernetes.md ("Handing models to LLMKube").
Dominant language
Go
Stars
232
Forks
37
Avg merge
21h 42m
Merged PRs (30d)
244

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from defilantech/LLMKube

All issues in defilantech/LLMKube

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.