Make Alembic the canonical SQLite migration path
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 28/100
调研方向
首先梳理 src/symphony_dbcli/store.py、models.py、cli.py、web/app.py、migrations/versions 和 scripts/provision-exedev-vm.sh。运行现有测试套件,并检查 Alembic 当前的 head 与 Store.SCHEMA 及修复路径的对应关系。完成的标准是:计划和实现覆盖全新及旧版 SQLite 数据库,使 init-db 具备幂等性,并在 systemd 服务启动前运行迁移。
由索引模型根据 Issue 内容生成。
描述
Goal
Make database schema changes predictable during VM deploys by moving schema ownership to Alembic and making deployment/startup run the migration path explicitly.
The repo already has Alembic scaffolding (alembic.ini, migrations/env.py, and migrations/versions/0001 through 0007), but the current runtime path still relies on custom startup repair code:
symphony-dbcli init-dbcallsStore.init()andcreate_model_tables(...).- FastAPI startup also calls
Store.init()andcreate_model_tables(...). Store.init()creates raw-SQL legacy tables and runs_migrate(...)for additive column repairs.create_model_tables(...)calls SQLAlchemyBase.metadata.create_all(...)and thenrepair_sqlite_model_tables(...).scripts/provision-exedev-vm.shrunsuv run symphony-dbcli --profile prod init-dbbefore installing/restarting the service, but the generated systemd unit does not run migrations on every restart.
We should make init-db the single blessed migration command, backed by Alembic, and then have the VM service run it safely before serve starts.
Non-goals
- Do not rewrite all
Storedata access to SQLAlchemy ORM as part of this issue. - Do not remove the existing SQLite DB or require manual data export/import.
- Do not change the configured DB location.
- Do not mix unrelated dashboard/runtime behavior changes into the migration work.
Phase 1: Schema inventory and migration plan
Produce a short implementation note or PR description that maps every existing schema owner to the future Alembic owner.
Inventory these areas:
- Raw SQLite tables in
src/symphony_dbcli/store.pySCHEMA. - Legacy repairs in
src/symphony_dbcli/store.py_migrate(...). - SQLAlchemy models in
src/symphony_dbcli/models.py. - SQLAlchemy repair entries in
_SQLITE_COLUMN_REPAIRS. - Existing Alembic migrations in
migrations/versions. - The
init-dbpath insrc/symphony_dbcli/cli.py. - The FastAPI startup path in
src/symphony_dbcli/web/app.py. - The generated systemd unit in
scripts/provision-exedev-vm.sh.
Progress checkpoint:
- A reviewer can tell which tables are already covered by Alembic and which tables still depend on
Store.init()/ repair logic. - The plan states whether each existing custom repair will be kept temporarily, moved into Alembic, or retired.
Phase 2: Make Alembic runnable from the app config
Make Alembic use the same database path as symphony-dbcli --profile ... init-db.
Expected work:
- Add a small internal migration runner or helper that resolves
WorkflowConfig.database.pathand runsalembic upgrade headagainst that SQLite DB. - Avoid requiring operators to manually construct
SYMPHONY_DATABASE_URL. - Keep this helper usable from tests without shelling out when practical.
- Ensure the migration command is idempotent.
Progress checkpoint:
- A test can run Alembic against a temporary empty SQLite DB using the same config path resolution that production uses.
- Running the migration command twice succeeds without changing data or failing.
Phase 3: Move SQLAlchemy-managed table changes fully under Alembic
Bring the SQLAlchemy model tables into a migration-first workflow.
Expected work:
- Verify the current Alembic head creates/updates the SQLAlchemy-managed tables to match the current models.
- Move any active
_SQLITE_COLUMN_REPAIRSentries for model tables into Alembic revisions where they are still needed. - Add tests that compare the schema from a fresh Alembic upgrade with the expected model columns/indexes.
- Decide how to handle existing DBs that may already have columns added by repair code before Alembic knows about them.
Progress checkpoint:
- Fresh DB +
alembic upgrade headyields the expected source/work-item/chat schema. - Existing DBs with previously repaired columns can still upgrade cleanly.
Phase 4: Cover legacy Store tables with handwritten Alembic migrations
Alembic does not require ORM models, so keep Store as the data access layer for now and add handwritten migrations for its raw-SQL tables.
Expected work:
- Create Alembic revisions for the legacy tables currently created by
Store.SCHEMA. - Create Alembic revisions for the additive repairs currently in
Store._migrate(...). - Preserve existing data when upgrading a populated DB.
- Add a test fixture for an older/minimal DB and verify upgrade to head adds missing columns.
Progress checkpoint:
- A fresh DB migrated with Alembic has both the legacy
Storetables and SQLAlchemy-managed tables. - A legacy DB missing columns such as worker heartbeat/deadline or PR cleanup columns upgrades to head successfully.
- No broad
StoreORM rewrite is required.
Phase 5: Make init-db the migration facade
Change symphony-dbcli init-db so operators use one command for schema setup and upgrade.
Expected work:
- Have
init-dbrun Alembic migrations to head. - Keep any required transitional compatibility behavior only where necessary and clearly mark it as temporary.
- Ensure workflow version loading still happens after schema is ready.
- Return clear errors if migration fails.
Progress checkpoint:
uv run symphony-dbcli --profile prod init-dbupgrades the configured DB to the latest schema.- The command remains safe to run repeatedly.
- Tests cover empty DB, current DB, and at least one older DB shape.
Phase 6: Update VM provisioning and systemd startup behavior
Make deploy/restart apply migrations before the app starts.
Expected work:
- Update
scripts/provision-exedev-vm.shso the generated systemd unit runs the migration facade beforeserve. - Prefer
ExecStartPre=<uv> run symphony-dbcli --profile prod init-dbif it behaves cleanly under systemd. - Keep
ExecStart=<uv> run symphony-dbcli --profile prod serve --no-reloadfocused on serving/runtime work. - Make failure behavior obvious: if migration fails, the service should not start with a mismatched schema.
Progress checkpoint:
- Re-running the provisioning script installs a unit that migrates on start/restart.
systemctl restart symphony-dbcli.serviceapplies pending migrations before launching the app.- Failed migrations leave useful logs in
journalctl.
Phase 7: Retire duplicate custom repair paths
After Alembic owns the schema, remove or narrow the old repair mechanisms.
Expected work:
- Remove redundant
_migrate(...)column additions that Alembic now owns, or keep only a guarded compatibility shim with a documented removal plan. - Remove redundant
_SQLITE_COLUMN_REPAIRSentries where Alembic now handles the change. - Avoid
Base.metadata.create_all(...)silently papering over missing migrations in production paths, unless intentionally kept for tests only. - Update docs so future schema changes require Alembic migrations.
Progress checkpoint:
- New DB columns are introduced through Alembic revisions, not hidden startup repair lists.
- Documentation tells future contributors where migrations live and how to test them.
- Quality gates pass:
uv run ruff check .,uv run ruff format --check .,uv run mypy, anduv run pytest.
Suggested validation matrix
- Empty SQLite DB ->
init-db-> app starts. - Existing local DB ->
init-db-> app starts and data remains readable. - Older schema fixture ->
init-db-> missing columns added and data preserved. - VM-style path/profile ->
init-db-> service start succeeds. - Running
init-dbtwice is a no-op. - Migration failure prevents service start and logs a clear error.
Open questions
- Should FastAPI startup keep running migrations directly, or should production rely on
ExecStartPreand fail fast if schema is stale? - Should legacy
Storetables be represented as SQLAlchemy CoreTablemetadata for schema clarity, without rewritingStoremethods? - Do we want a one-time Alembic
stamppath for existing databases that have already been repaired by startup code?
- 主要语言
- Python
- 星标
- 0
- 派生
- 0
- PR 合并指标
- 30 天内没有已合并 PR
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
相似的 Issue
-
area: harness bug status: needs-triage
难度 2/5 1-3 小时 新手友好度 75/100
Human-Agent-Society/reef#625 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 1/5 1 小时以内 新手友好度 80/100
learningequality/kolibri#15351 · 2 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
-
Name consistency 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
eellak/triplestore#65 · 1 条评论 ·