D5.1 Map DTC course platform data to the shared apps

First confirm C5.3 is marked done in docs/plan/STATUS.md, then read docs/plan/phase-5.md section D5.1, AGENTS.md, and _docs/PROCESS.md. Map the listed course and coursework models, rehearse the P6 migration and route changes, and verify _docs/compatibility/course-route-contracts.json, uv run pytest -q, and matching counts; certificates and their URLs must remain preserved.

Plan issue D5.1, phase 5 of the community-base unification plan.

Spec: docs/plan/phase-5.md, sections D5.1 and DTC development data boundary, in DataTalksClub/community-base (PR #322, merged as 3449643dc5061676586911efb4a9c0c7d3e7a2c6). Status is tracked in docs/plan/STATUS.md there, which stays the source of truth.

Depends on: C5.3, D5.3a. Do not start until both are done in STATUS.md. D5.3a is the source/policy milestone under DataTalksClub/website#436 and depends on C5.4 and C7.12b; this issue does not waive the tagged-package, donor or AISL compatibility gates.

Process

This repository's AGENTS.md and _docs/PROCESS.md govern this work, with _docs/specs/ as the product authority. The plan says what to build and how to verify it; the site process says how work is done here.

Development data boundary

Before implementation, name the exact target and choose fresh or in-place adoption. The owner-authorized fresh exception applies only to an explicitly disposable DTC development logical database; #440's immediate target is dtc_website_dev. It does not authorize production, AISL, Relay, or shared RDS resets. Fresh mode need not preserve historical database IDs or rows from the discarded development schema; mark those preservation checks Not applicable: authorized fresh DTC development target, never passed. In-place mode retains the lossless migration and development-copy rehearsal gates. Both modes preserve every supported course feature, public route, API, permission, Studio interaction, and visible presentation. Fresh import must establish stable authored identities so subsequent syncs and source moves preserve newly created rows and learner links.

Spec

Repository: DataTalksClub/website. Depends on: C5.3, D5.3a.

D5.3a supplies the verified source/policy contract. This issue owns storage adoption, field/data
mapping and repointing existing callers while preserving current flat routes and UI. Prove
nested/YAML-homework persistence and identity at the target-model boundary using shared APIs;
new nested reader projection acceptance is D5.3b and repository-reader retirement is D7.3.
Activate one shared-parser-backed scheduled/webhook writer against accepted package storage on
the selected development target with approved compatible development source refs. Source updates
must keep working: no legacy jobs writing old/disconnected tables, mirrored writes or second
runtime parser. D7.3 completes six-source registration/proof and generic reader retirement;
D7.4 alone changes final live authored refs under its original freezes. C5.3, all donor gates
and the selected-mode data boundary remain unchanged.

Steps

1. Map every course, cohort, curriculum, enrollment, progress and coursework field and workflow to the shared apps, a DTC extension, or an approved source import. Keep LearnerProfile and DTC-owned cohort placement, access, registration, routes and presentation.
2. Fresh mode: migrate empty PostgreSQL storage with tagged package releases and import approved public sources from all six DTC course repositories. Cover flat and nested/YAML homework, source/question identities, source order, cohort dates and project references. Every course has a cohort; a course without a source-defined offering gets one persisted synthetic self_paced cohort with null start/end dates. Repeat import must not duplicate rows. Inventory public content absent from those repositories, including project prerequisites and historical offerings; source sync alone does not reproduce the full public catalog. Define the intended fresh target public-content inventory and an explicit disposition for old-only content. Required target content and references need an approved public source and compatible importer before adoption can finish. A missing import path for a supported feature blocks completion. Do not recreate discarded historical rows merely to match old counts.
3. Prove the target-model contract before using the separately controlled development public bootstrap. Its audited artifact imports public data only; its current command is not assumed compatible with package storage. Use synthetic records for protected learner flows. Do not populate the target from protected CMP or production exports.
4. In-place mode: retain P6 mapping and P14 development-copy rehearsal, including course_family_catalog.py where needed. Preserve existing learner rows, scheduled cohort dates, certificate URLs and identities; prove reversibility and equal before/after counts.
5. Re-point internal callers while preserving _docs/compatibility/course-route-contracts.json, APIs, permissions, redirects, Studio and visible behavior. Delete courses, studio_courses, course_management, cadmin, review_import and compatibility code only after a complete caller, command, task and route inventory proves each removed part has no live responsibility.

Verification

• Fresh mode: migrations and drift checks pass from zero. Record safe public source/import counts against the defined target inventory, required public relationships, zero courses without cohorts, repeat-import results and stable-identity evidence. Synthetic learner tests do not replace required public source/import coverage.
• In-place mode: historical row, cohort-date and certificate-URL preservation, reversibility and count equality pass on the development copy.
• Both modes: synthetic enrollment, progress, homework save/submit/score/review, projects, peer review, leaderboard, registration, certificates and self-paced access pass. Route/API/Studio and rendered behavior match existing contracts. uv run pytest -q and the site's required verification gates pass, followed by a green development deployment.

Required cohort coverage (linked shared-package contract)

The shared curriculum contract is tracked in community-base#295. Include it in the D5.1 mapping and selected-mode proof; D5.2 verifies it after cutover.

• Every imported or migrated DTC Course has at least one associated Cohort.
• Preserve every scheduled cohort selected for the target and its source-defined dates. If a course has no source-declared offering, create one synthetic self-paced cohort with mode=self_paced and null start/end dates. Do not infer or invent dates.
• Re-import and rerunning the mapping are idempotent. Keep identities and learner links created after a fresh import stable across source resync and directory moves; in-place mode additionally preserves pre-existing learner rows.
• Record course and cohort counts against the reviewed target inventory, plus the count of courses lacking a cohort, which must be zero. Historical before/after equality applies only to in-place mode.

Sequencing references

• Source/policy proof: D5.3a / #446.
• Storage and single source-writer activation: D5.1 / #414.
• Reader projection: D5.3b / #447.
• Full preservation acceptance: D5.3 / #436.
• Coordinated course cutover: D5.2 / #415.
• Plan sequencing correction: community-base #348; its phase-5/phase-7 and generated STATUS update must merge before these revised dependencies govern implementation. No milestone is completed by this grooming.
• Package conversion semantics: C7.12b / community-base #347. Package capability, donor compatibility and site adoption remain separate gates.