Enforce SQLAlchemy-only database connections and Alembic-only schema changes
Los mantenedores suelen responder en 3 días
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 35/100
Línea de trabajo
Start with the existing rules registry and pytest guards, then inspect policyengine_api/data/v2/catalog/publication_staging.py to understand the two stated exceptions. Add a guard covering the specified code paths, proof fixtures, CI wiring in the existing test job, and guidance in docs/engineering/skills/alembic-migrations.md. Done means the guard rejects direct relational driver use and non-Alembic persistent schema changes, accepts only the exact exceptions, and its tests run in CI. Pull request #3979 is already open.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem
The Public API opens every relational database connection through SQLAlchemy today: API v1 builds its Cloud SQL engine in policyengine_api/data/orm.py (the Cloud SQL connector is passed to create_engine as creator=), and API v2 builds its SQLModel engine from a postgresql+psycopg URL in policyengine_api/data/v2/database.py. Schema changes go through the autogenerated Alembic revisions under migrations/v1 and migrations/v2.
Nothing enforces either of these. The rules registry has related rules, but none of them forbids:
- a direct driver connection such as
psycopg.connect(...),pymysql.connect(...)orsqlite3.connect(...), or use of the driver connection beneath SQLAlchemy, in application code, scripts or CI code.api.data.runtime-sql-confinedonly confines raw SQL to the data-access layer, andapi.services.no-direct-queriesonly covers services; - a schema change outside Alembic, such as
metadata.create_all(engine),Table.create(...),op.*outsidemigrations/, orCREATE/ALTER/DROP/TRUNCATESQL text.api.migration.alembic-autogenerated-onlyis doc-only, andapi.migration.alembic-v1-qualified/api.migration.alembic-v2-qualifiedcheck the revisions, not the code around them.
Both have happened before: the Stage 12 provisioning script (removed in d8bd72191) opened psycopg connections directly, and the local database bootstrap added in 6c7804100 called V1Base.metadata.create_all at runtime.
Proposed rule
Register an enforced rule, api.data.sqlalchemy-and-alembic-only: code connects to relational databases only through SQLAlchemy engines, connections and sessions, never by calling a database driver directly, and changes persistent schema only through Alembic revisions.
- Enforce it with a pytest guard that parses non-test code (
policyengine_api/,scripts/,migrations/,.github/, excluding tests and the generated revisions undermigrations/*/versions/). - Declare the two current exceptions exactly, both in
policyengine_api/data/v2/catalog/publication_staging.py:COPY ... FROM STDINthrough the psycopg connection beneath a SQLAlchemyConnection, because SQLAlchemy has no COPY API;STAGING_METADATA.create_allforTEMPORARYtables that Postgres drops at commit, which are not persistent schema.
- The guard should fail if either exception grows or is no longer needed.
- Redis (
policyengine_api/runtime_cache/) is a key-value cache, not a relational database, and is out of scope. - Add proof fixtures, CI wiring through the existing
testjob, and a section indocs/engineering/skills/alembic-migrations.md.
- Lenguaje dominante
- Python
- Estrellas
- 18
- Forks
- 33
- Merge medio
- 2 d 15 h
- PR fusionados (30 d)
- 35
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de PolicyEngine/policyengine-api
-
Dificultad 1/5 1-3 horas Aptitud para principiantes 88/100
PolicyEngine/policyengine-api#3911 ·
Los mantenedores suelen responder en 3 días
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 82/100
PolicyEngine/policyengine-api#3910 ·
Los mantenedores suelen responder en 3 días
-
Let a saved reform or household point at a new v2 policy or household without changing its IDPosiblemente ocupada @anth-volk la tomó hoy. Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 68/100
PolicyEngine/policyengine-api#4004 ·
Los mantenedores suelen responder en 3 días
-
Releases do not check that each environment calls its own simulation servicePosiblemente ocupada @anth-volk la tomó hoy. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 46/100
PolicyEngine/policyengine-api#4002 ·
Los mantenedores suelen responder en 3 días
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 18/100
PolicyEngine/policyengine-api#4000 ·
Los mantenedores suelen responder en 3 días
Todos los issues de PolicyEngine/policyengine-api
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
CLIMADA-project/climada_python#1337 ·
Los mantenedores suelen responder en 3 días
-
tail without reverseAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
8go/matrix-commander#233 ·
-
documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100
-
documentation enhancement priority:high size:M
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Los mantenedores suelen responder en 1 día
-
Validate non-divisible effective batch sizes in the Unsloth LoRA backendPosiblemente ocupada @debesh-biswas la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100