Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Generated schemas go stale because each source writes into the shared tree on its own

Abierto
  • ¿Algo no está bien?
#428 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 3 días

@stevendborrelli ya está trabajando en esto.

Desde el 15/10/2026.

  • #430 de @stevendborrelli — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
18/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
go, python, typescript
Área
build-system, cli

Línea de trabajo

Start in internal/project/build.go around lines 269 and 277, where dependency and API schemas are written in two separate passes, then read Generate and Add in internal/schemas/manager/manager.go and the lock format in lock.go. Reproduce by removing an XRD under apis/ and building, to see the old model stay in schemas/. Done means that removed models disappear, the lock records sources, languages and a generator fingerprint, and unchanged groups skip regeneration. The issue sets the design, so confirm the approach with the maintainers before coding.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement
What problem are you facing?

Generated schemas go stale because each source writes into the shared tree on its own. This ends up resulting in a case where users are running rm -rf schemas before builds.

Summary

crossplane project build writes the language schemas under schemas/ (Go,
Python, KCL, JSON) one source at a time. Each source's output is copied on top
of the existing tree, and nothing is ever removed. The lock file records only
each source's version, not the tree as a whole. As a result, the tree on disk
can keep models that no source produces anymore, or miss models the project
should have, and the build never notices.

I found this while adding TypeScript support (#170), which can't work at all
with the current design. I'd like to fix it in its own PR first so the
behavior change can be reviewed separately from TypeScript.

Current impact (on main today)
You do this What happens Workaround today
Rename or delete a kind in an XRD under apis/ The old kind's model stays in schemas/<lang>/ forever Delete schemas/ by hand
Remove a dependency from crossplane-project.yaml All of its models stay, and the lock keeps its entry crossplane dependency clean-cache
Add a language to spec.schemas.languages Dependencies don't get models for the new language. Each dependency's version is unchanged, so it is skipped crossplane dependency clean-cache
Remove a language Its directory stays Delete it by hand
Delete one language directory (e.g. schemas/python/) Dependencies aren't regenerated for it. The lock still says they're current crossplane dependency clean-cache
Build with no changes at all apis/ schemas are regenerated anyway (wasted time on every build, run and render) None

None of these cause an error. Users see wrong or leftover models in their
editor and in function builds, and the fix ("clean the cache") isn't obvious.

This happens in real projects. Modelplane now deletes schemas/ before
every build to work around it (modelplaneai/modelplane#170, commit
a3e9ba9,
June 2026). The commit message says:

Both generators are also additive: they overwrite the files they emit but
never remove models or lock entries for XRDs or dependencies that have
been dropped or renamed. Stale schemas and lock entries accumulated across
regenerations. This is how schemas/.lock.json came to carry entries for
provider versions the project no longer referenced.

The workaround costs a full regeneration on every build: about 100s for
modelplane's 13 providers, even when only an XRD changed.

TypeScript (#170) is fully blocked. Its generator writes one npm package per
run. The root index.js, package.json and the shared _schemas/ folder
describe everything in that run. When each source runs on its own, each one
rewrites those root files to describe only itself, and the last one wins. Only
the last source's models can be imported.

Where the problem is in the code

Line numbers are from main @ 833d6e0.

  1. Two separate write passes in the build.
    internal/project/build.go:269 calls dependencyManager.AddAll (one write
    per dependency). Then build.go:277 calls schemaManager.Generate for
    apis/ (one more write). Nothing ever sees all sources at once.

  2. Writes only add files, never remove them.
    internal/schemas/manager/manager.go:72 (Generate) copies the generator
    output into schemas/<lang>/ with CopyFilesBetweenFs (lines 131 and 135).
    Nothing clears the old output first. This causes the leftover kind and
    leftover dependency rows in the table.

  3. Freshness is checked one source at a time.
    manager.go:51 (Add) skips a source if its recorded version matches. It
    never checks the language set or whether the directories still exist. The
    dependency manager calls Add at internal/dependency/manager.go:226 and
    :379-383. This causes the language rows in the table.

  4. The lock only stores versions.
    internal/schemas/manager/lock.go:22 has just
    Packages map[string]string. It can't record which languages were
    generated, or whether the tree on disk was produced from exactly these
    sources.

  5. apis/ is never cached.
    build.go:277 calls Generate (always regenerates) instead of Add. The
    version hash for apis/ is computed and written to the lock, but nothing
    ever reads it back.

Proposed fix

Collect every source first, then generate in two cached groups and swap the
finished tree into place.

Two source groups
  • Dependencies: every source from crossplane-project.yaml, including
    transitive xpkg dependencies.
  • APIs: the project's own apis/ directory.

Each group is generated, cached and checked for freshness on its own. An XRD
edit regenerates only the APIs group; the dependency output is reused from the
cache. A dependency change regenerates only the dependencies group, which is
rare.

Within a group, sources are generated once per source type (CRD and OpenAPI).
A generator whose output describes the whole run (TypeScript) can implement a
small schemaMerger interface to combine parts into one tree. Other
generators copy parts one after another, and the copy fails on a conflict (see
below).

Lock file (internal/schemas/manager/lock.go)
{
  "version": 2,
  "generator": "<fingerprint>",
  "languages": ["go", "python"],
  "groups": {
    "dependencies": { "xpkg://...": "sha256:..." },
    "apis": { "fs://apis": "<hash>" }
  }
}

A group is fresh only when all of these hold:

  • its source map exactly matches the current sources,
  • the language set matches,
  • the generator fingerprint matches,
  • its cached output exists.

This one rule covers every row in the impact table:

  • A removed dependency changes the source map.
  • A changed language list changes languages.
  • A deleted directory fails the "cached output exists" check.

The generator fingerprint is built from the CLI version, the generator
image digests and the generator options, such as the Go runtime-object and
accessor settings. A CLI upgrade that changes generated code then regenerates.
Today it doesn't.

An old lock has no version field, so it reads as stale and the project
regenerates once. An unreadable lock is treated as stale rather than failing
the build. Today getLock returns the decode error, so a lock damaged by an
interrupted write breaks every later build.

Lenguaje dominante
Go
Estrellas
20
Forks
34
Merge medio
5 d 4 h
PR fusionados (30 d)
21

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de crossplane/cli

Todos los issues de crossplane/cli

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.