Derive a declared runtime topology (which service is configured to reach which) and export architecture diagrams

Start by reading the dependency context in #94 and #54, then inspect pkg/deployables/ and the proposed pkg/topology/ package. Use tests/topology/e2e.py, tests/topology/fixtures/, and the listed metamorphic and unit tests to define the implementation scope. Done means deterministic connects_to and contract edges with evidence, valid Mermaid, DOT, and JSON exports, and the stated precision and recall gates.

User outcome

dircue draws the architecture diagram a new engineer wishes existed, from declarations alone, with evidence for every arrow:

• which services exist;
• what each one is built from;
• where each one runs;
• which service is configured to talk to which;
• through which declared ports, protocols and contracts;
• what datastores, caches and queues each one is wired to.

Exportable as Mermaid, Graphviz DOT and JSON.

Why

• "How does it work?" is half of dircue's mission question. Once #94's review fixes land (the PR currently emits no runs edges; see its P0-5 item and #54), builds/runs edges will connect deployables to components. But the most useful architectural fact is still missing: who is configured to call whom.
• That fact is usually declared statically. For example, microservices-demo's Kubernetes manifests set PRODUCT_CATALOG_SERVICE_ADDR: "productcatalogservice:3550" on the frontend. Other common sources:
  • Compose depends_on, links and environment URLs;
  • Helm values;
  • Kubernetes Service selectors and ports;
  • Ingress and Gateway routes;
  • service-mesh routes;
  • connection strings in application configuration.
• Diagrams drawn from real declarations, where every edge is clickable back to a file and line, are rare. Hand-drawn architecture docs go stale, and dircue deliberately doesn't trust documentation. "Your architecture, derived from your manifests, with receipts" demonstrates the evidence discipline in a way anyone can see at a glance.

Proposal

• New edge type (a schema proposal): connects_to, from a component/deployable/workload to another workload/service/capability. Properties: protocol, port and declared target. Basis declared_config, with the source key and span as evidence. Each edge is qualified:
  • resolved: the target name matches a declared Service or Compose service in the same map;
  • external: a hostname outside the map;
  • unresolved: templated or env-indirected.
• Resolution rules (static, bounded, documented per rule):
  • Kubernetes Service name/namespace DNS forms (svc, svc.cluster.local) → Service → selector → workload;
  • Compose service names and aliases;
  • host:port and URL-shaped env or config values;
  • Ingress/Gateway backends.
  • Helm templating is resolved only for literal values from values.yaml defaults, and recorded as conditional.
• Contracts: link protobuf/OpenAPI/GraphQL contracts (#56) to the components that implement them (generated server stubs, handler registration imports) vs consume them (client stubs). This replaces the "same .proto copied into four services" noise #94 showed with implements/consumes edges.
• Exports:
  • dircue map diagram --format mermaid|dot|json map.json, reading only the saved map;
  • deterministic layout hints: stable ordering and grouping by namespace or compose project;
  • collapses large graphs by component or namespace.

Tasks

• Schema proposal for connects_to and implements/consumes contract edges (a #74 amendment), with semantics documentation.
• Kubernetes/Compose/Helm-defaults address extraction and resolution.
• Contract implementation/consumption linking for protobuf (Go, Java, Node, Python, C# stubs) and OpenAPI where it's statically clear.
• Diagram exporters, with golden outputs.
• A summary section: "Topology: 11 services, 14 declared connections (12 resolved, 2 external)".

Test plan

Unit tests — pkg/deployables/ and new pkg/topology/ (new package)

• Kubernetes env-var address extraction: table-driven tests on literals of the forms svc, svc.ns, svc.ns.svc.cluster.local and host:port; assert correct service_name, namespace and qualification (resolved / external / unresolved). Include env keys with non-URL-shaped values (must produce no edge) and keys whose values are Go template expressions {{ .Values.addr }} (must produce unresolved, not resolved).
• Compose address extraction: depends_on, links, environment URL values; assert service-name-only references resolve to a declared compose service and URL-shaped values pointing outside the file produce external.
• Helm literal-only rule: values from values.yaml defaults resolve; any {{ template ... }} call in an env value must produce unresolved, not silently swallow the rest of the file. (This is the "Helm names swallow the rest of the template file" regression from the PR #94 review.)
• Implements/consumes contract edges: implements vs consumes is decided only by each service's generated server/client stubs and registration or import evidence, never by where a .proto file sits. Byte-identical copies of one contract in several service directories (the microservices-demo pattern) are recognized as one contract (same content ID), with per-copy evidence, rather than N unrelated contracts.
• Comment and doc suppression: URL-shaped strings in Markdown, YAML comments and inline comments must produce no edge. Document the rule in a test-table column "in_comment": true.
• Fuzz target FuzzTopologyExtraction in pkg/topology/fuzz_test.go (new file): feed arbitrary YAML bytes; assert no panic, no resolved edge to a name absent from the declared services list, deterministic output. Tie to #87 for corpus persistence.

Independent oracle / ground-truth fixtures — tests/topology/fixtures/ (new directory)

Tie to #75 and #84.

• Hand-labeled topology sets (committed, not fetched):

  • microservices-demo-k8s/: a minimal reproduction of the Kubernetes env-var pattern (PRODUCT_CATALOG_SERVICE_ADDR: "productcatalogservice:3550" and three more frontend→backend pairs). Labels record every expected connects_to edge with source key, line, qualification and target service.
  • eshop-compose/: an eShop-style Aspire/Compose app with depends_on and environment URL entries; at least two resolved and one external edge.
  • helm-chart/: a chart whose values.yaml supplies a literal backend address, with a separate template that renders it; the literal must produce resolved, the raw template expression must produce unresolved.
  • negative-fixtures/: env vars whose values are doc strings, comments, or {{ .Env.FOO }}-style indirections; all must produce zero edges.

• Precision/recall gates: for each labeled set, precision ≥ 0.95 and recall ≥ 0.90 on connects_to edges. Gate enforced in tests/topology/e2e.py.

• Pinned real repositories (on demand): hand-label the full declared topology of microservices-demo, eShop and one Helm-based app at pinned commits (labels under new tests/topology/labels/, repos fetched by the harness into .cache/, never committed). Apply the same gates on workflow_dispatch, and feed the results into the #107 accuracy cards.

• Fixtures are kept small (< 15 files each) and committed. The 21-repo public corpus from tests/map_corpus/fetch_public.py is not used here; topology assertions on public repos belong in an on-demand extension of that corpus. Tie to #88.

Diagram validity — tests/topology/e2e.py (new file), container-only

Tie to #84. Render Mermaid and DOT outputs inside a dedicated pinned container (new tests/topology/Dockerfile, run with --network none), with pinned tool versions recorded in a PROVENANCE.json under tests/topology/:

• Mermaid: @mermaid-js/mermaid-cli (mmdc) at a pinned npm version; assert exit 0 and a non-empty SVG output.
• DOT: Graphviz dot at a pinned version; assert exit 0 and a non-empty output.
• These rendering checks run in CI only on workflow_dispatch (diagram rendering is slow and requires a container build). Mermaid and DOT syntax validity without rendering is checked by a lightweight Go parser unit test on every PR.

Metamorphic invariants — tests/topology/metamorphic.py (new file)

Tie to #85.

• Adding a no-op Kubernetes Service with no env-var references must not change any existing edge.
• Reordering env vars within a manifest must not change the edge set.
• Renaming a service and updating all referencing env vars must produce the same graph structure with updated names.
• A connects_to edge must never appear without a service_name and a source span in its evidence.

Mutation-testing focus

Tie to #86. Prioritise the qualification-resolution logic (the resolved / external / unresolved branch) and the comment-suppression guard; both have conditions that can be silently flipped by a mutation.

Performance and resource checks

Tie to #89 and #90.

• On every PR touching pkg/deployables/ or pkg/topology/: deterministic edge-count counters asserted by a unit test (TestTopologyEdgeCounts on the committed fixtures).
• On-demand (make bench-topology, new target): run tests/topology/e2e.py over the public corpus from tests/map_corpus/fetch_public.py and record wall time, RSS, edge counts and precision/recall to JSONL.

E2E script — tests/topology/e2e.py (new file)

Runs on every PR touching pkg/deployables/ or pkg/topology/. JSONL log fields per fixture: fixture, command, wall_seconds, connects_to_total, resolved_count, external_count, unresolved_count, precision, recall, missing_edges (list), extra_edges (list), mermaid_exit, dot_exit. Human summary includes a per-fixture precision/recall table. Exits non-zero on gate failure or diagram render error (diagram rendering gated to workflow_dispatch only).

CI placement

• Every PR touching pkg/deployables/, pkg/topology/, pkg/mapbuild/ or tests/topology/: unit tests (including fuzz seed runs), edge-count assertions, and a fast Mermaid/DOT syntax check (no container). Add a topology job to ci.yml or to a new topology.yml (triggers: push, pull_request, workflow_dispatch; no schedule:).
• workflow_dispatch / make topology-e2e only: container-based diagram rendering and on-demand public-corpus precision/recall.

Regression fences

• A test that asserts {{ .Values.x }}-shaped Helm strings produce unresolved, never resolved, blocking the "names swallow the template" regression.
• A test that asserts byte-identical .proto copies collapse to one contract identity, while each service's implements/consumes edge still comes only from its own stub evidence.
• A test that asserts no edge is emitted for any source with "in_comment": true in the negative-fixtures table.

Risks

• False edges from generic URL heuristics. Restrict extraction to declared keys and known shapes; everything else stays unresolved.
• Overclaiming. Declared topology is not runtime traffic. The wording must stay "configured to connect", never "calls".

Success criteria

• microservices-demo's diagram matches its labeled topology exactly.
• Every edge links to evidence.
• Unresolved addresses are listed, not hidden.

Related: #54, #56, #23, #74, #80, and #94 (P0-5). Related idea issues: #104, #102.

Sequencing: Depends on #94's builds/runs edges (review item P0-5) and #54. It feeds #104.

---

This proposal came out of an idea-wizard session on 2026-09-23. It isn't a release commitment, and the milestone is deliberately unset so the owner can decide per idea. It must preserve dircue's contracts:

• deterministic, offline output, with no execution of inspected content;
• the legacy Linguist-compatible CLI and output unchanged;
• unknown never rendered as absent;
• no provider findings or verdicts imported;
• CI event-driven only (no schedule:).