Lockfile + semantic diff for the identities agent configs run as (MCP, Actions, compose, K8s, Terraform) — feedback welcome

Read the registry's documentation for server authors and the server.json declarations, then compare them with GrantLock's SPEC.md and the assumptions listed in this issue. The issue asks whether a committed grantlock.json checked by repository CI would complement publish-time validation and whether the assumptions match listed servers. Done means maintainers have answered both questions; no registry files or tests are named as change targets.

Hi! I've been working on GrantLock, a small tool that records who an AI agent stack runs as — API keys, OAuth grants, service accounts, MCP server secrets, CI tokens and cloud roles, with their scopes — in a committed lockfile (grantlock.json), and then fails CI on semantic changes to those grants: a scope added to an existing identity, a wildcard/admin grant, a new identity in production, a long-lived key where OIDC is available, a grant nobody uses any more. It never records secret values (names, providers, storage class and a fixed-table format class only), and the scanner makes no network calls.

Why I'm posting here: in a recent thread in this repository (#1783, about capability records for listed servers), it came up that the registry validates entries at publish and edit time and does not re-check them on a schedule (the consolidated tracker for that gap is #1715). The same gap exists one layer down, for identity. What a listed server runs as — the secrets its configuration expects, the scopes behind them, the tokens in its own CI and release workflows — lives in the server's repository, and can change there at any time without anything noticing. A committed lockfile is meant to make that drift visible in the repository where it happens, checked by that repository's own CI.

As a field check during development (Phase 2 of the project), I ran GrantLock on a sibling repository, modelcontextprotocol/servers, at commit 3638028 (shallow clone; scan → lock generated from the repository itself → check). Because the lock comes from the repository itself, only state findings can appear, not changes. GrantLock reported 42 findings, and I reviewed all of them by hand: 42 valid, 0 false positives, 0 undecided. By surface: 41 came from the 11 GitHub Actions workflow files (39 identity bindings) and 1 from the MCP config (.mcp.json, 1 binding). By class: 40 are G4 (no owner — under GrantLock's definition an owner is a CODEOWNERS entry or a declared owner, and no CODEOWNERS rule covers the workflows or the MCP config), and 2 are G5 (one Anthropic API key secret used by two jobs of the same workflow, where per-job credentials would be possible). The MCP server configuration examples in 11 Markdown files are in README code blocks, which GrantLock does not read, so those were not covered by the scan.

How it compares to secret scanners. On the GrantBench development corpora, gitleaks and trufflehog find literal secrets (G1: 6/10 and 7/10 on dev, 5/6 and 6/6 on dev-ext) but 0 of the value-free classes G2–G8 — they look for secret values, and a widened permission or a shared token contains none.

Honest status. On the development corpora GrantLock detects every planted class with no clean false positives, but those corpora were written by the same author, so they are consistency checks. Against sealed holdouts written by independent authors, the pre-registered gate (G2/G3/G7 recall each ≥ 80%, mean ≥ 90%, clean false positives ≤ 5%) has not been passed:

• holdout v1: G2 6/7, G3 5/7, G7 8/8 (mean 85.7%), clean false positives 0/16 → failed on recall; four blind spots fixed afterwards
• holdout v2: G2 7/8, G3 7/8, G7 8/8 (mean 91.7%), clean false positives 5/16 → failed on false positives (one over-broad compose rule); fixed afterwards and re-verified on the now-disclosed corpus (0/16 false positives, diff 8/8) — which is not a pass

Known limitations are listed in the README (nested Terraform maps, toolsets set via an MCP HTTP header, service-account key files mounted from a Kubernetes Secret volume, MCP literals under credential names).

Two questions for the maintainers, if you have a moment:

1. Would a committed grantlock.json in a listed server's own repository — kept honest by a grantlock check step in that repository's CI — be a useful complement to the registry's publish-time validation? For example, something the registry's documentation for server authors could point to, so that a widened permission, a new secret or a token shared across jobs becomes visible after listing, in the repository where the change happens.
2. Do the assumptions in GrantLock's SPEC.md match how listed servers declare what they run as? The registry's server.json already declares packages and environment variables; GrantLock assumes credentials are referenced through environment variables (${VAR}) rather than literals, that toolsets/scopes are declared in env or CLI arguments, that each Actions job's GITHUB_TOKEN is treated as its own identity, and that ownership is expressed through CODEOWNERS. If the grants behind the declared variables usually live somewhere else, I'd like to know where a lockfile should look.

Code, spec, decision log and gate reports: https://github.com/NagaYu/grantlock
Benchmark (development splits only; sealed holdouts are not published): https://huggingface.co/datasets/NagaYu/grantbench

The repository is published for evaluation under All Rights Reserved; I'm happy to discuss licensing.