oclapi2: one unattended command that runs the PR checks in an isolated local stack and cleans up after itself

Abierto
  • ¿Algo no está bien?
#2,973 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
38/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
docker, docker-compose, elasticsearch, github-actions, postgresql, python
Área
devops, infrastructure, testing

Línea de trabajo

Read .github/workflows/pr.yml, docker-compose.ci.yml, coverage.sh, and AGENTS.md to trace the current CI commands, service setup, and local instructions. Start by comparing the workflow steps with the acceptance criteria, then identify how the gate can run them against the mounted worktree and clean up its isolated stack. Done means the command, fast module form, safe cleanup, and updated AGENTS.md meet the listed criteria.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

component/api decision/planning-needed signal/low-risk signal/well-specified stage/triaged type/infra

Why

To run oclapi2's PR checks locally you need a Postgres and Elasticsearch stack and a ~21 GB image, and a full run takes up to ~45 minutes. AGENTS.md tells an agent to docker exec into the developer's running oclapi2-api-1 container. That's fine with a person at the keyboard, but not for an agent session working a ticket unattended in its own git worktree:

  • Wrong code under test. The shared container runs another checkout's code, so the tests don't exercise the worktree's change.
  • Shared state. Two sessions at once share one database and one set of search indexes.
  • No cleanup. Nothing removes what a session started.

The FHIR work (OpenConceptLab/ocl_online#451) has run the checks by hand with a five-command docker compose recipe. The recipe works, but it is copied by hand, and it never cleans up: dozens of leftover gate volumes sit on the shared dev Mac now.

This ticket adds one command to this repo that runs the checks unattended and safely, so that agent sessions can take oclapi2 tickets end to end. ocl_issues#2883 is waiting on it.

What the command must do

  1. One command, no prompts.

    • It runs from any worktree and needs no TTY.
    • It exits non-zero on any failure.
    • It ends with a one-line summary and prints where the coverage report is.
    • A fast form runs one module's tests (e.g. core.fhir) for the edit-and-test loop.

  2. It runs what .github/workflows/pr.yml runs, in the same order and with the same environment. As of 4b331a0b, that's:

    • pylint -j0 core/;
    • manage.py search_index --create;
    • the tests without the es tag, with TEST_ES=off and --parallel=4;
    • the es-tagged tests, with --parallel=1;
    • coverage combine, then coverage report --include=core/* --fail-under=97;
    • all with ENVIRONMENT=ci, PYTHONHASHSEED=2 and COVERAGE_CORE=sysmon.

    pr.yml changed three times on Oct 26, so a copy of its commands would drift. Either pr.yml and the gate call one shared script, or a test fails when their commands differ. coverage.sh could become that script, but today it's out of date: one serial run with a 95% minimum.

  3. It tests the worktree's own code. It bind-mounts the current directory, as docker-compose.ci.yml already does. It never docker execs into another stack, and never re-tags a shared image from one worktree's build.

  4. It's isolated.

    • Its own compose project, oclapi2-gate-<name>, and no container_name:.
    • No host ports: it uses run --rm.
    • Service images pinned to CI's: postgres:14.7 and elasticsearch:8.15.2.
    • It waits on health checks, not sleeps.
    • On OrbStack, it runs orb start first.

  5. It cleans up. It runs down -v when it finishes, pass or fail. A separate cleanup command removes stacks that a crash left behind. That command:

    • matches only a label the gate puts on its own stacks, never the oclapi2-* name prefix;
    • never touches a volume labeled ocl.keep=true (seeded content that takes hours to rebuild), or the developer's own oclapi2 and oclapi2-api-* projects;
    • never runs machine-wide (no docker system prune or docker volume prune).

  6. No secrets and no outside services. It reads no .env and no ~/.ocl, and makes no calls to staging or production. Compose reads a .env in the project directory by default, so the gate has to turn that off.

  7. It shares the image. It uses the dependency-only arm64 image oclapi2-local:deps-<hash>, where <hash> is the first 12 hex digits of sha256(requirements.txt + Dockerfile), and builds it only when it's missing.

    • The build script that exists today moves into this repo, with the same tag formula, so existing images and their other users keep working.
    • The image has no ENVIRONMENT=ci and no test-only entrypoint baked in. The gate sets them, so the same image can also run a long-lived local API.

  8. It doesn't compete for memory.

    • It refuses to start while a corpus seed is loading the local stack, for example by checking a lock file the seed holds.
    • It never runs pylint and the tests at the same time: the ES tests flake under load.

Docs: add a "Run the checks unattended" section to AGENTS.md. The docker exec advice can stay for interactive work, but unattended sessions use the gate. The same edit fixes the PR base branch: AGENTS.md says to branch from develop (GitFlow), but only master exists.

Acceptance criteria

  • A full run on a warm image passes on master's head in about 45 minutes or less, and agrees with CI on the same commit: the same pass or fail, and coverage within 0.1%.
  • The fast form runs a single module.
  • Gates on two worktrees at once share no database, index or container, and both clean up.
  • A run killed halfway leaves a stack that the cleanup command removes. A test shows the cleanup leaves ocl.keep=true volumes and other projects alone.
  • A change to a pr.yml test step either reaches the gate as well (shared script) or fails a test.
  • The AGENTS.md section is written, and its base branch is corrected.

Out of scope

  • The long-lived local API and the FHIR parity tooling from ocl_online#451. They reuse the image and nothing else.
  • Changing what CI checks.
Lenguaje dominante
Sin datos de lenguaje
Estrellas
4
Forks
2
Merge medio
2 h 12 min
PR fusionados (30 d)
1

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de OpenConceptLab/ocl_issues

Todos los issues de OpenConceptLab/ocl_issues

Issues similares

Más issues de DevOps

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.