Post-cutover: migrate body-heavy entities to gitsheets v1.2 content-typed (markdown) records
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Refactor
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- markdown, typescript
- Domain
- api, backend, databases, documentation
Research direction
Start with the schemas in packages/shared/src/schemas/ and the .gitsheets/.toml configurations, then inspect apps/api/src/store/memory/loader.ts for body-less and detail reads. Trace the serializers, FTS pipeline, import-laddr.ts, scrub-data.ts, and the planned migrate-to-content-typed.ts script. Done means the listed entities use the new markdown format, reads and indexes handle lazy bodies, existing data migrates safely, and both specifications are updated.
Written by the indexing model from the issue text.
Description
gitsheets v1.2.0 added content-typed records — sheets opt into format.type = 'markdown' to store records as .md files with TOML frontmatter and a designated body field. Plus lazy body loading via query({ withBody: false }).
This is the biggest one-time upgrade we'd take from the gitsheets 1.x line. Not urgent — defer to after cutover-prep ships so we're not refactoring entities mid-migration.
Why migrate
- Snapshot is actually-readable. Contributors cloning
codeforphilly-data-snapshotsee real.mdfiles in any markdown viewer — instead of parsing TOML records to find the prose. - Authoring via PR. Staff / maintainers can edit a project overview in any markdown editor and PR it; currently they roundtrip through the API.
- Listing performance.
queryAll({ withBody: false })on hot paths: projects-index, activity feed, FTS seeding, snapshot scrub. - Indexes stay fast. Index builds use body-less reads natively in v1.2.
What changes
Entities with substantial body content:
Project—overview(markdown body),summary(short markdown) → migrateoverviewas the body field, keepsummaryin frontmatterProjectUpdate—body(markdown) → migratebodyas the body fieldProjectBuzz—summary(markdown) → migrate as bodyPerson—bio(markdown) → migrate as bodyHelpWantedRole—description(markdown) → migrate as bodyTag—description(markdown, short) → optional; cheaper to leave as TOML field
The migration is bounded; entities without long bodies (ProjectMembership, SlugHistory, Revocation, TagAssignment, HelpWantedInterestExpression) stay as TOML records.
Tasks
- Schema reshape in
packages/shared/src/schemas/— one designated body field per content-typed entity (rename or restructure the existingoverview/body/bio/description/summaryfields). - Update
.gitsheets/<sheet>.tomlconfigs with[gitsheet.format] type = 'markdown' body = '<fieldName>'. - In-memory loader in
apps/api/src/store/memory/loader.ts— use{ withBody: false }for index-building reads; lazy-load viaSheet.loadBody(record)when serving record detail responses. - Serializers in
apps/api/src/services/serializers/—*Html/*Excerptderived from the body field instead of the legacy string field. - FTS pipeline in
apps/api/src/store/fts.ts— body included in the indexed text via lazy-load batch. apps/api/scripts/import-laddr.ts— write the new markdown format for migrated entities.apps/api/scripts/scrub-data.ts— the snapshot now contains real.mdfiles; verify the scrub still strips PII correctly across the new file shape.- The data repo's existing TOML records need migration once — write a one-shot
apps/api/scripts/migrate-to-content-typed.tsthat reads existing records and rewrites as.mdper the new format. - Update
specs/behaviors/markdown-rendering.mdandspecs/data-model.mdto reflect content-typed entities.
Why defer
cutover-prepis next and depends on every other plan; this would invalidate frozen plans (storage-foundation,read-api,write-api,laddr-import,public-snapshot-scrub).- The benefit is real but landing is post-cutover work, not pre-cutover refactor.
Out of scope
gitsheets checkpre-commit hooks belong in the data repo, not this code repo.- The bundled Claude Code skill at
node_modules/gitsheets/skills/gitsheets/is available once we bump the dep range; future plans touching gitsheets can load it.
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 1
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 25
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from CodeForPhilly/codeforphilly-ng
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
MarkdownEditor toolbar: use Radix Toolbar from radix-ui instead of the hand-rolled roving tabindex Openenhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
All issues in CodeForPhilly/codeforphilly-ng
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
ontola/atomic-server#1625 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
melgarafael/DeskcommCRM#1451 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
-
bug via-triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
bot:ai-assisted component:compact-js status:untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
midnightntwrk/midnight-sdk#403 ·