Export sync data to an archive file with a disk connection
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 10/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Da chiarire
- Stato di attività
- Attiva
- Ambito
- backend, databases, distributed-systems
Direzione di ricerca
Read the linked design spec and implementation plan, then review the dependent sub-issues from #362 through #387. Start with morango/sync/ and MorangoProfileController.create_disk_connection(path), along with morango/models/core.py and morango/constants/settings.py. Done means all sub-issues are closed and the characterization, contract, parity, and full SQLite/PostgreSQL suites pass.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.
Overview
This tracking issue groups the work for Phase 1 of disk sync. Phase 1 adds a disk connection that exports a fresh SQLite archive through the existing sync stages.
Background & Motivation
Morango syncs only over the network today. MorangoProfileController.create_disk_connection is a stub. Large Kolibri instances hold many facilities, and some facilities are not active all the time. Infrastructure managers need to export the data of a facility to an archive file and store it offline.
The design adds a shared Connection interface with a transport-neutral Peer protocol. Network and disk connections both implement it. The sync stage operations become shared Peer* operations, so that one set of client-side logic drives both connection types.
An archive holds the same morango data that a network server holds after it receives the same push. An archive also holds a manifest. This parity lets later phases reuse the existing receiver code.
Design: spec. Plan: implementation plan.
User Story
As someone who manages the infrastructure for large Kolibri instances with many facilities,
I want a way to export archives of facilities,
So that I can store the data offline until it is needed again.
This work covers the export only. Import from an archive is Phase 2. Kolibri can already remove the data of a facility from an instance, so that step is not part of this work.
Description & Expected Outcomes
The work lands in milestones. The refactor milestones (M1 to M4) come first. They change no behavior, and the characterization suite must pass without edits after each one. The feature milestones (M5 and M6) come next, and M7 adds the final verification.
| Milestone | Plan task | Issue | Blocked by |
|---|---|---|---|
| M0 Spike | 1 | #362 | none |
| M1 Characterization | 2 | #363 | none |
| M1 Characterization | 3 | #364 | #363 |
| M2 Database handle | 4 | #365 | #362, #364 |
| M2 Database handle | 5 | #366 | #365 |
| M2 Database handle | 6 | #367 | #366 |
| M2 Database handle | 7 | #368 | #365 |
| M3 Connection seam | 8 | #369 | #364 |
| M3 Connection seam | 9 | #370 | #369 |
| M3 Connection seam | 10 | #371 | #369, #370 |
| M3 Connection seam | 11 | #372 | #371 |
| M4 Shared operations | 12 | #373 | #370 |
| M4 Shared operations | 13 | #374 | #368, #371, #373 |
| M4 Shared operations | 14 | #375 | #374 |
| M5 Archive | 15 | #376 | #375 |
| M5 Archive | 16 | #377 | #376 |
| M5 Archive | 17 | #378 | #362, #365, #377 |
| M5 Archive | 18 | #379 | #378 |
| M5 Archive | 19 | #380 | #378 |
| M5 Archive | 20 | #381 | #378 |
| M5 Archive | 21 | #382 | #377, #378, #379 |
| M6 Disk connection | 22 | #383 | #370, #373, #380, #381, #382 |
| M6 Disk connection | 23 | #384 | #367, #371, #383 |
| M7 Verification | 24 | #385 | #372, #384 |
| M7 Verification | 25 | #386 | #375, #384 |
| M7 Verification | 26 | #387 | #386 |
Deliverables & Contracts
When all sub-issues close, morango delivers these capabilities:
MorangoProfileController.create_disk_connection(path)returns aDiskSyncConnection.- A push through a disk connection writes a complete archive at the path.
NetworkSyncConnectionandDiskSyncConnectionimplement oneConnectioninterface.- The
Peer*operations contain the client-side stage logic for both connection types. - The old
Network*operation names,server_info, andserver_cert=keep their behavior and emit aDeprecationWarning.
Acceptance Criteria
- All sub-issues are closed.
- The characterization suite passes without edits after each refactor milestone.
- The contract suite passes for the network and the disk connection.
- The parity test passes: the archive data equals the data of a network server for the same push.
- The full test suite passes on SQLite and PostgreSQL.
Technical Pointers & Architecture
- Target Components / Context:
morango/sync/(connections, contexts, operations, database handle, archive),morango/models/core.py,morango/constants/settings.py. - Related Patterns: The middleware dispatches operations by context type (
morango/registry.py). The design reuses this dispatch for connection-specific overrides. - Data Model & Schema Considerations: One new model,
ArchiveManifestEntry, with migration0006. The migration only adds a table. - Resilience & Failure Modes: If an export fails, the connection discards the partial archive. A lock file prevents two exports to the same path.
Notes & Tradeoffs
- Out of scope: Import from an archive (Phase 2), resume of disk sync sessions (Phase 3), and updates to an existing archive (Phase 4).
- Kolibri companion changes: See spec §3.9. Kolibri routers must allow morango migrations on aliases that start with
morango_archive_. Kolibri settings overrides must list thePeer*operations. - Follow-up issue: Separate commands from queries in the Peer protocol. The spec lists this item in §8.
- Security: An archive holds facility data in plain text. An archive never holds private keys.
Metadata
- Complexity: High
- Target Branch: release-v0.9.x
AI Usage
Drafted with Claude (Claude Code) from the approved design spec and implementation plan. The author reviewed the requirements, and the code references were checked against the release-v0.9.x codebase.
- Lingua principale
- Python
- Stelle
- 15
- Fork
- 23
- Merge medio
- 1g 11h
- PR unite (30g)
- 4
Preparare l'ambiente
- Include un Dockerfile o un file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di learningequality/morango
-
DEV: dev-ops DEV: distributions DOCS: developer
Difficoltà 1/5 1-3 ore Idoneità per principianti 78/100
learningequality/morango#387 ·
-
DEV: backend DEV: dev-ops TAG: unit tests
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
learningequality/morango#385 ·
-
DEV: backend P0 - critical TAG: tech update / debt TAG: unit tests
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
learningequality/morango#364 ·
-
DEV: backend TAG: unit tests
Difficoltà 4/5 3-5 giorni Idoneità per principianti 55/100
learningequality/morango#386 ·
-
DEV: backend P0 - critical TAG: new feature
Difficoltà 4/5 3-5 giorni Idoneità per principianti 38/100
learningequality/morango#384 ·
Tutte le issue di learningequality/morango
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 Mezza giornata Idoneità per principianti 70/100
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
Qiskit/qiskit-ibm-runtime#3431 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
[Lesson] A compatibility-gate rejection is a verdict, not something to overwrite with --accept-riskApertalesson-submission needs-ac pending-review
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
Ikalus1988/MisakaNet#2870 ·
I maintainer di solito rispondono entro 1 giorno
-
feature:LinkChecker
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
digitalfabrik/integreat-cms#4594 ·
I maintainer di solito rispondono entro 5 giorni