rockets-repository-firestore: parity plan to make Firestore a production store of record (transactions, uniqueness, zod compiler, cursors, TTL)
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 28/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- firebase, typescript
- Domain
- backend-api-design, database
Research direction
Start with the parity matrix, packages/rockets-repository-firestore/README.md, and src/utils/firestore-repository.util.ts, then inspect define-firestore-repository.ts, firestore-repository.module.ts, and firestore-repository.ts. Map the missing transaction, compiler, uniqueness, cursor, projection, batch-write, and TTL capabilities to the existing interfaces. Done means a prioritized, scoped parity plan that distinguishes implementable adapter work from Firestore limits and identifies the required tests or deployment artifacts.
Written by the indexing model from the issue text.
Description
Firestore adapter: what is missing to reach TypeORM-level parity as a production store of record
Draft issue for
conceptadev/rockets. Formatted for
.github/ISSUE_TEMPLATE/feature_request.yml(sections Use case and Proposal);
everything else is supporting detail that can be collapsed or dropped.Every claim was verified against the code at the paths given, in this repository. Where a
commonly repeated assumption about the adapter turned out to be wrong, it is corrected
in "Assumptions corrected" rather than repeated as a gap.
Suggested title: rockets-repository-firestore: parity plan to make Firestore a
production store of record (transactions, uniqueness, zod compiler, cursors, TTL)
Labels: package: rockets-repository-firestore, type: enhancement, priority: high
Use case
Field evidence
We rebuilt a real production API on Rockets SDK v8 as a benchmark — 51 wire endpoints, 65
frozen error codes, ~28.7k LOC of server source, twice (once against a frozen HTTP
contract, once contract-free). The application it replaces is Firestore-native: chat
sessions, CMA PDF records, video records, rate-limit buckets and an analysis cache all live
in Firestore today.
We still chose TypeORM, and the decision was made and documented before the first line of
code was written. It is the single largest architectural concession of the study.
Consumer repository referenced throughout (private):
realtystack, branch
feat/rockets-api-max. Absolute path on the machine where this was verified:
/Users/thiagoramalho/Documents/Thiago/Workspace/Concepta/OpenSource/realtystack.
Paths below are writtenrealtystack/<repo-relative path>.
realtystack/docs/benchmark/build-plan.md§1 (lines 19-52) — "Decision: TypeORM as the
single app repository adapter … Firestore is not used", with five numbered reasons: the
zod compiler being TypeORM-only, transactions, cursor pagination, retention/TTL, and the
SQLite test tier.realtystack/docs/benchmark/00-gap-register.mdrow 1 (line 7) — "zodResourcecompiler is
TypeORM-only (typeOrmZodEntityCompiler); no Firestore compiler" — and row 2 (line 8), the
Firestore→SQL swap with its (contract-invisible) divergences.
What that cost. The headline measured result of the whole study is 19 of 52 routes
generated (37%), up from 5 of 47 (10.6%) on the frozen branch
(realtystack/docs/benchmark/rockets-build-metrics.md:343-358). All 19 come from
zodResource, which requires a SchemaEntityCompiler. There is exactly one implementation
in this repository and it emits TypeORM decorators. Had we chosen Firestore, the number
would have been 0 — the framework's headline capability is currently unavailable to
Firebase-first users, i.e. precisely the audience the package names in
packages/rockets-repository-firestore/README.md:50 ("You want a Firebase-first app with
Firebase Auth + Firestore storage").
The README is already honest about part of this (README.md:52-56: "When NOT to use this
package: You need ACID transactions across multiple entities"). This issue argues that (a)
the constraint is broader than cross-entity ACID, (b) most of it is reachable with
Firestore's own primitives — runTransaction, startAfter, FieldValue.increment, batched
writes, native TTL policies, deterministic document ids — and (c) a small, clearly-labelled
residue is a genuine limit of the database, not of the adapter, and should be stated as a
contract rather than left implicit.
The ask
Define and close the parity set that lets an application select
defineFirestoreRepository() as its root adapter — not as a per-entity side store — and
get the same framework capabilities a TypeORM app gets today.
Parity matrix — TypeORM vs Firestore, today
Legend: ✅ full · 🟡 partial · ❌ absent · ⛔ not achievable in Firestore's model (see
"Honest limits").
| # | Capability | TypeORM | Evidence (TypeORM) | Firestore | Evidence (Firestore) |
|---|---|---|---|---|---|
| 1 | zod → entity compiler (SchemaEntityCompiler) |
✅ | packages/rockets-repository-typeorm/src/zod/compile-entity.ts:73-116; exported at the /zod subpath (src/zod/index.ts) |
❌ | No /zod subpath in packages/rockets-repository-firestore/package.json exports:9-16; no entityCompiler on the bootstrap (src/integration/define-firestore-repository.ts:19-35). Seam exists and names Firestore as the easy case: packages/rockets-core/src/common/repository/schema-entity-compiler.interface.ts:26-31, .../repository-module.interface.ts:17-20 |
| 2 | Transaction factory registered on the feature module | ✅ | @concepta/nestjs-repository-typeorm/dist/typeorm-repository.module.js:20-26 (transactionFactories), dist/transaction/typeorm-transaction.js |
❌ | packages/rockets-repository-firestore/src/firestore-repository.module.ts:27-33 and src/utils/firestore-repository.util.ts:56-68 return a DynamicRepositoryModule with no transactionFactories |
| 3 | ctx on repo calls joins the ambient transaction |
✅ | @concepta/nestjs-repository-typeorm/dist/repository/typeorm-repository.js:28-41 (getRepo(ctx) → trx.getOrStart(...).getClient()), used by every method (:162-216) |
❌ | Every do* method takes _options and never reads it: src/repository/firestore-repository.ts:97,119,129,139,150,159,169,182. FirestoreBackend has no transaction handle at all (src/interfaces/firestore-backend.interface.ts:34-51) |
| 4 | transactional: true on a generated CRUD op |
✅ | packages/rockets-core/src/domain/interfaces/rockets-resource-definition.interface.ts:218 → @concepta/nestjs-crud/dist/infrastructure/utils/get-transactional-decorators.js → Transactional() |
🟡 (silently inert) | The decorator applies, but with no factory TransactionManager.isSupported is false and TransactionScope.run defaults to propagation: 'SUPPORTS' → the operation runs non-transactionally with no error (@concepta/nestjs-repository/dist/transaction/transaction-scope.js) |
| 5 | TransactionScope usable from app services |
✅ | Same as #2/#3 | ❌ | Same as #2/#3 — silent no-op |
| 6 | Hooks that read/write through the repo inside the operation | ✅ | getRepo(options.ctx) above; hook resolution @concepta/nestjs-core/dist/infrastructure/hook/hook.resolver.service.js |
🟡 | Hooks run, but the transactional half of { ctx } is unconditionally discarded (#3). See consumer gap realtystack/docs/benchmark/00-gap-register.md row 51 (line 62) for how sharp this footgun already is on SQL |
| 7 | Soft delete + restore + withDeleted |
✅ | @DeleteDateColumn via compile-entity.ts:153-156; adapter honours withDeleted |
✅ (with caveats) | src/repository/firestore-repository.ts:167-190; withDeleted at :211. Caveats: column auto-detection is heuristic (src/repository/firestore-entity-metadata.ts:60-83) and the exclusion is an in-memory post-filter that disables every server-side fast path (see item 11 and P1-4) |
| 8 | Unique constraints from db: { unique: true } |
✅ | packages/rockets-repository-typeorm/src/zod/compile-entity.ts:100-109,173-175; meta declared at packages/rockets-core/src/zod/fields.ts:29-32,67-68 |
❌ | src/repository/firestore-entity-metadata.ts:11-51 builds columns and reads only the soft-delete flag; unique/index are never consulted anywhere in the package |
| 9 | Indexes from db: { index: true } / composite indexes |
✅ | compile-entity.ts:102-108,178-180 (@Index) |
❌ | Never read; no index artifact is emitted. Firestore's composite-index requirement therefore surfaces only in production as FAILED_PRECONDITION |
| 10 | Filter operators executed natively | ✅ 17/17 | @concepta/nestjs-repository-typeorm/dist/repository/typeorm-repository.js:88-120 — EQ, NE, GT, GTE, LT, LTE, CONTAINS, NCONTAINS, STARTS, NSTARTS, ENDS, NENDS, IN, NIN, IS_NULL, NOT_NULL, BETWEEN all become SQL |
🟡 6/17 | Native: EQ, NE, GT/GTE/LT/LTE, IN (≤30), array-contains, prefix-STARTS as a range (src/repository/firestore-where.translator.ts:244-313). In-memory post-filters: NIN, IS_NULL, NOT_NULL(partial), CONTAINS/NCONTAINS, NSTARTS, ENDS/NENDS, BETWEEN (:139-243). 'not-in' is declared in src/interfaces/firestore-query.interface.ts:9 but never emitted |
| 11 | count / findAndCount |
✅ | TypeORM count/findAndCount |
🟡 | Implemented, and uses the server-side aggregation query.count().get() (src/backends/admin-firestore.backend.ts:150-152) — but only when the branch has zero post-filters, which a soft-deletable entity never satisfies (src/repository/firestore-query-runner.ts:86-104, admin-firestore.backend.ts:134-148) |
| 12 | Offset pagination pushed to the store | ✅ | skip/take → SQL OFFSET/LIMIT |
🟡 | limit(skip + take) + local slice (src/backends/admin-firestore.backend.ts:123-131) — same caveat as #11: skipped entirely when post-filters exist (:108-121, "read all matching docs and slice locally") |
| 13 | Keyset / cursor pagination | ❌ | — | ❌ | Neither adapter has it, because RepositoryFindOptions has no cursor field (@concepta/nestjs-repository/dist/repository/interfaces/repository-options.interface.d.ts). Firestore is the adapter where this is cheapest to add (startAfter), and it is currently unused (grep -rn startAfter packages/rockets-repository-firestore/src → 0 hits) |
| 14 | Projection (select) |
✅ | typeorm-repository.js:154 (select: options.select) |
❌ | Dropped: src/repository/firestore-repository.ts:203-214 forwards only where/order/skip/take/withDeleted |
| 15 | Relations / join |
✅ | typeorm-repository.js:132-149 (translateJoin, resolveJoinClauses) |
⛔/❌ | Silently ignored; metadata.relations is hardcoded [] (src/repository/firestore-entity-metadata.ts:49). Relational joins are not a Firestore feature — but silently returning unjoined rows is an adapter defect. See "Honest limits" |
| 16 | Atomic multi-row writes | ✅ | One SQL statement / one transaction | ❌ | doCreateMany loops doCreate (src/repository/firestore-repository.ts:105-114), doDeleteMany loops doDelete (:157-165); documented as non-atomic in README.md:196-199. WriteBatch/BulkWriter unused |
| 17 | Atomic counter / compare-and-set | ✅ | Inside a transaction | ❌ | doUpdate/doUpsert/doReplace are set(..., {merge}) of the caller's in-memory copy — last write wins (src/repository/firestore-repository.ts:116-146). No FieldValue.increment, no update() precondition, no version column |
| 18 | Native retention / TTL | 🟡 | Not a TypeORM feature either; apps use a deleteAt column + sweep |
❌ | No TTL story (grep -rn "ttl|TTL" packages/rockets-repository-firestore/src → 0 hits) — even though Firestore has native TTL policies and TypeORM does not. This is a place Firestore could beat parity |
| 19 | Schema lifecycle | ✅ | synchronize/migrations flow through TypeOrmModule.forRoot({...connection, entities}) (packages/rockets-repository-typeorm/src/define-typeorm-repository.ts:24-29) |
⛔ (n/a) | Firestore is schemaless — no migration equivalent is needed. The real analog is index + TTL policy deployment, which is item 9/18, not a gap in itself |
| 20 | Per-entity collection/table naming | ✅ | Entity(options.table) |
✅ | collection flows through the planner (packages/rockets-core/src/infrastructure/resource/planner/repository-plan.ts:80-91) into src/utils/firestore-repository.util.ts:37 |
| 21 | Root-adapter usage (repository: on the bundle) |
✅ | defineTypeOrmRepository(connection) |
🟡 | Structurally supported (defineFirestoreRepository implements RepositoryBootstrap), but items 1-3, 8, 16-17 make it unusable as the only adapter for a non-trivial app. The README itself scopes the package as "per-entity opt-in, not a wholesale replacement" (README.md:24-27) |
| 22 | Test tier parity | ✅ | Everything runs on SQLite in the default suite | 🟡 | InMemoryFirestoreBackend filters and sorts everything in memory and enforces none of Firestore's query rules (src/backends/in-memory-firestore.backend.ts:74-89); the emulator suite is opt-in and covers value semantics only (vitest.firestore.config.mts → **/*.emulator-spec.ts; src/__tests__/firestore-backend.emulator-spec.ts) |
Score today: 5 ✅ / 8 🟡 / 7 ❌ / 2 ⛔ across 22 capabilities.
Definition of "production-ready" (the objective bar)
This issue proposes that "Firestore is production-ready" means all five of the following
are demonstrably true in this repository's own test suites:
(a) Root adapter + full generated CRUD. An app can pass defineFirestoreRepository() as
the bundle's repository and use zodResource with every operation generated
(list/read/create/update/replace/delete/restore), with schema-derived DTOs and
OpenAPI — i.e. the "19 of 52 generated routes" result is reproducible on Firestore, not only
on TypeORM.
(b) The four transactional behaviours run without touching the raw SDK. Taken verbatim
from the consumer API, because they are the general shapes every real app needs:
| Behaviour | Consumer implementation (SQL today) | What it needs |
|---|---|---|
| Fail-closed rate limit — read N buckets, deny consuming nothing, else increment all, in one unit | realtystack/api-rockets/src/rate-limit/rate-limit.store.ts:94-141 |
transaction + atomic increment + unique bucket id |
| Idempotent enqueue — deterministic UUIDv8 reservation, fingerprint replay/409/expired-404, dispatch, rollback on unconfirmed dispatch | realtystack/api-rockets/src/resources/video/application/commands/handlers/enqueue-video.handler.ts:53-58,134-151 |
transaction spanning a create + an external call |
| Chat turn lock — reserve/complete/clear with replay-before-staleness and 180 s takeover | realtystack/api-rockets/src/resources/chat-sessions/chat-session-store.service.ts:60,128,170 |
transaction + compare-and-set |
| Render lease — single winner, expired-lease takeover, renewal | realtystack/api-rockets/src/resources/video/application/commands/handlers/video-lease.handlers.ts:65-86,124-139 |
transaction + compare-and-set |
(c) Generated cursor list works. A keyset page is expressible through
RepositoryFindOptions and compiles to startAfter on Firestore. (This one needs a core
change too — see item 13 of the matrix.)
(d) Native retention. A deleteAt-style field can be declared as a Firestore TTL field,
stored as a real Timestamp, and the policy is emitted as a deploy artifact rather than
hand-maintained.
(e) Suite parity, not just API parity. The Rockets e2e/repository suites that run against
SQLite run against the Firestore emulator with the same test set — not a reduced
value-semantics subset. Today vitest.firestore.config.mts includes only
**/*.emulator-spec.ts, and the single such file tests value ordering and null/missing
semantics. Query legality (composite indexes, the orderBy/inequality rule), transaction
limits and write-batch limits are invisible to CI, which is why a green build can fail on
first contact with real Firestore.
Honest limits — where Firestore cannot have parity, and the contract we propose instead
Being precise about this matters more than the feature list: some items above are adapter
gaps (fixable), others are database limits (must become an explicit contract).
| Limit | Why it is real | Proposed contract |
|---|---|---|
| Relational joins (matrix #15) | Firestore has no server-side join. Emulating it is N+1 reads with no transactional consistency across the fan-out | Do not emulate silently. Either (i) fail at boot when an entity registered on Firestore declares a join/relation the adapter cannot serve, naming the relation; or (ii) support it explicitly through the existing federation orchestrator (@concepta/nestjs-repository/dist/federation/*) with the read amplification documented. Denormalisation stays the app's choice, but the framework must not pretend the join happened |
| Multi-field unique constraints (matrix #8) | Firestore's only uniqueness primitive is the document id | Two tiers: (1) single unique column → map it to the document id (deterministic id). This covers the majority in practice — 3 of the consumer app's 4 unique columns (bucketId, documentId, uid) are natural document ids. (2) anything else (composite or secondary unique) → a uniqueness-index collection ({collection}__unique__{fields}, value-derived doc id holding the owner row id) written with create() inside the same transaction. If neither is configured, fail at boot naming the column — never silently drop it |
| Read-after-write inside a transaction | Firestore requires all reads before all writes in a transaction, and may retry the whole closure | The imperative TransactionInterface (start/commit/rollback, @concepta/nestjs-repository/dist/transaction/interfaces/transaction.interface.d.ts) does not express either constraint. Proposal in P1-1: expose a callback-shaped runInTransaction(ctx, fn) capability alongside the imperative bridge, and document that handlers must be idempotent and must not interleave reads after writes. A bridge that hides this would produce transactions that are wrong under contention |
| Transaction/batch size | ≤ 500 writes per batch; 270 s transaction lifetime; getAll argument limits |
Chunk in the backend where semantics allow (batch), and throw a typed adapter exception where they do not (a single transaction exceeding 500 writes cannot be split without losing atomicity) |
OR across fields |
Firestore's or() is limited and disjunctions across different fields need index support; the adapter already fans out DNF branches client-side (src/repository/firestore-query-runner.ts:20-53) |
Keep DNF fan-out, but make the read amplification measurable (branch count in a debug log) and cap it in line with RepositoryAdapter.MAX_DNF_BRANCHES |
| Schema migrations (matrix #19) | Schemaless store — nothing to migrate | Not a gap. The equivalent deliverable is index + TTL policy emission (P2-3) |
Everything not in this table is an adapter gap and is in scope for the phases below.
Proposal — three delivery phases
Phase 1 — unblock "store of record" (correctness)
Nothing here is optional for an app that writes concurrent state. All four consumer
behaviours in criterion (b) fail silently today.
P1-1 · Transactions (matrix #2, #3, #4, #5) — effort L
The extension point already exists in the contract:
DynamicRepositoryModule.transactionFactories?: TransactionFactoryDescriptor[]
(@concepta/nestjs-repository/dist/interfaces/repository-module.interface.d.ts), so this is
additive for core; only FirestoreBackend changes shape.
- Add to
FirestoreBackend(src/interfaces/firestore-backend.interface.ts):
runTransaction<T>(fn: (tx: FirestoreTransactionHandle) => Promise<T>): Promise<T>and an
optionaltxargument onget/set/create/delete/queryBranch.AdminFirestoreBackend
implements it withgetFirestore().runTransaction(...);InMemoryFirestoreBackend
implements it with a copy-on-write snapshot plus a conflict check on commit. - Register a
TransactionFactoryDescriptorfromcreateFirestoreFeatureModule
(src/utils/firestore-repository.util.ts:56-68), mirroring
@concepta/nestjs-repository-typeorm/dist/typeorm-repository.module.js:20-26. - Thread
options.ctx→TransactionManager→ backendtxin everydo*method
(src/repository/firestore-repository.ts), the way
@concepta/nestjs-repository-typeorm/dist/repository/typeorm-repository.js:28-41does. - Resolve the shape mismatch honestly (see "Honest limits"): ship the callback-shaped
capability first, and only then the imperative bridge if it can be made safe. - Until a factory exists,
transactional: trueon a Firestore-backed op should refuse at
boot, not run unprotected. A silent no-op is worse than an unsupported feature.
P1-2 · Atomic increment and compare-and-set (matrix #17) — effort M
- A sentinel understood by
toStore(src/repository/firestore-repository.ts:254-267) and
byAdminFirestoreBackend.serialise(src/backends/admin-firestore.backend.ts:228-230),
translated toFieldValue.increment(n)/FieldValue.serverTimestamp()and emulated
numerically in the in-memory backend. - A
preconditionon the backend'sset/delete(lastUpdateTime, must-exist,
must-not-exist), surfaced as an optionalexpectedVersiononRepositoryUpdateOptions.
Without P1-1 this is the only way to make a lease or a lock correct on Firestore.
P1-3 · Uniqueness (matrix #8) — effort M (tier 1: S)
Implement the two-tier contract from "Honest limits". create() already surfaces the right
error for the document-id tier (FirestoreDuplicateIdException,
src/backends/admin-firestore.backend.ts:65-81). Boot-time refusal for anything unenforceable.
P1-4 · Soft delete must stop forcing full-collection reads (matrix #7, #11, #12) — effort M
augmentBranchesForSoftDelete (src/repository/firestore-query-runner.ts:86-104) appends a
soft_delete_excluded post-filter to every branch whenever the entity has a soft-delete
column and withDeleted !== true. Post-filters are exactly what disables the fast paths:
pushToServer requires postFilters.length === 0 (firestore-query-runner.ts:22-25),
queryBranch skips limit() when post-filters exist (admin-firestore.backend.ts:108-121,
comment: "read all matching docs and slice locally"), and countBranch abandons the
aggregation for a full read (:134-148).
Net effect: on any entity with dateRemoved/deletedAt — the Rockets standard — every
list and every count reads the whole filtered collection into the Node process. The
limit(skip + take) optimisation advertised in CHANGELOG.md is unreachable for exactly
those entities. On Firestore that is a billing curve, not an exception.
Fix: the post-filter exists because Firestore cannot express "field is null or
missing". Solve it at write time — always materialise the soft-delete column as an explicit
null in toStore, so where(field, '==', null) becomes a real server-side filter and the
branch keeps postFilters.length === 0. Ship a documented backfill for pre-existing
documents and keep the post-filter only as an opt-in compatibility mode.
P1-5 · Reconcile orderBy with inequality filters (matrix #10) — effort M
buildOrderedQuery (src/backends/admin-firestore.backend.ts:208-220) applies the caller's
orderBy verbatim. Firestore requires a query with a range/inequality filter to order by
that field first. The consumer app's retention hook produces deleteAt > now
(realtystack/api-rockets/src/resources/video/video-expiry-scope.hook.ts:38-46) while the
list ops order by updatedAt desc — a combination Firestore rejects at runtime. The same
applies to NOT_NULL, which the translator maps to !=
(src/repository/firestore-where.translator.ts:211-216).
Fix: prepend the branch's inequality fields to the pushed orderBy and re-sort locally
only when the requested order differs — or refuse at translation time with a message naming
the required composite order. Either way, emit the composite-index requirement (P2-3).
P1-6 · Batched writes (matrix #16) — effort S
FirestoreBackend.writeBatch(ops) over db.batch() (chunked at 500) for Admin and
apply-all/discard-all for in-memory; route doCreateMany/doDeleteMany through it, joining
the ambient transaction when one is active.
Phase 1 acceptance criteria
- A handler using
TransactionScope.run(ctx, …)with{ ctx }on repo calls executes
atomically against the Firestore emulator;propagation: 'MANDATORY'throws when it cannot.- The four behaviours of criterion (b) pass their existing e2e suites on a Firestore
backend: deny-consumes-nothing; replay 201 / conflict 409 / expired 404 with rollback on
unconfirmed dispatch; replay-before-staleness with 180 s takeover; single-winner lease
with expired takeover and renewal failure.- A concurrent-writers emulator test proves increment and compare-and-set (no lost update,
exactly one lease winner out of N racers).db: { unique: true }is enforced or refused at boot with the column name.ownerUid == x AND deleteAt > now,order updatedAt desc,take 20runs as a
server-side query with a pushedlimit, andcount()uses the aggregation — asserted by
read counts against the emulator, not by returned rows.createManyis all-or-nothing within Firestore's batch limits, or throws a typed
exception when it cannot be.transactional: trueon a Firestore-backed op never silently degrades.
Phase 2 — feature parity
P2-1 · firestoreZodEntityCompiler (matrix #1) — effort M-L
Ship @concepta/rockets-repository-firestore/zod. The class itself is trivial — the
interface doc says as much (packages/rockets-core/src/common/repository/schema-entity-compiler.interface.ts:26-31:
"Firestore needs little more than a named class token plus a collection name") — but that
sentence is what makes this deceptively easy. The real content is the metadata the
adapter currently guesses or ignores:
- Soft-delete column. Today auto-detected by instantiating the class and walking
prototype property names (src/repository/firestore-entity-metadata.ts:60-83) — which
returns nothing for a compiled class whose fields are decorator-only or unassigned, and
thendelete()throws at runtime (src/repository/firestore-repository.ts:225-232).
FirestoreProviderOptions.softDeleteFieldexists
(src/interfaces/firestore-provider-options.interface.ts:12) but is never set by the
planner —buildRepositoryPlanforwards onlykey,entityandcollection
(packages/rockets-core/src/infrastructure/resource/planner/repository-plan.ts:80-91),
confirming the README note atREADME.md:170-179. Fix both: carry it in compiled metadata
and make the existing override reachable viaModuleResourceEntityEntry. db.unique/db.index(P1-3, P2-3).- Date typing. The TypeORM compiler maps
z.iso.datetime()to a realdatetimecolumn
(packages/rockets-repository-typeorm/src/zod/compile-entity.ts:279-281); the Firestore
compiler must decide per field whether the stored form is aTimestamp(required for
native TTL, P2-2) or an ISO string, andtoStore/fromStoremust honour that. - Warning from the field: a compiler that emits "little more than a named class token"
lands squarely in a known DTO-layer failure —isBaseEntityResponseFieldreturnsfalse
whenmeta.dbisundefined, so a schema with no persistence metadata is invisible to the
response DTO and shipsnullcolumns. Documented with reproduction at
realtystack/docs/benchmark/00-gap-register.mdrow 45 (line 56).
P2-2 · Native TTL (matrix #18) — effort M
Firestore has per-collection TTL policies keyed on a Timestamp field — a capability
TypeORM does not have, so this is a chance to exceed parity rather than chase it. Expose
ttlField on the entity registration row, store it as a real Timestamp, and emit the
policy into the deploy artifact (P2-3). The consumer app currently implements retention as a
read-time deleteAt > now filter on every find plus an app-owned sweep
(realtystack/api-rockets/src/resources/video/video-expiry-scope.hook.ts); its Firestore
predecessor marked 2 sites rockets-gap: ttl (see Prior art).
P2-3 · Emit index and TTL artifacts (matrix #9) — effort M
Generate a firestore.indexes.json fragment from the registered entities plus the queries
the adapter can build (owner + range + order combinations, plus P1-5's composite orders),
as a build step or a defineFirestoreRepository({ emitIndexes }) option. This repo already
carries firebase.json and firestore.rules at the root, so there is a natural home.
P2-4 · Relations: explicit, not silent (matrix #15) — effort M
Implement the "Honest limits" contract: refuse at boot or serve through the federation
orchestrator. Ending the silent-drop is the deliverable; full join support is not.
P2-5 · Projection (matrix #14) — effort S
Push options.select to query.select(...); keep full reads on the document-id path and
document why.
P2-6 · Native not-in; make post-filter cost visible (matrix #10) — effort S
Emit not-in (≤10 values) natively — the op is already declared
(src/interfaces/firestore-query.interface.ts:9) and never used. For the operators that must
stay in memory, add a debug log of the scan, and an opt-in strictPushdown mode that throws
instead of scanning.
Phase 2 acceptance criteria
bindZodResources(firestoreZodEntityCompiler)produces the same generated route set as
the TypeORM compiler for at least one full resource (list/read/create/update/delete/restore),
with schema-derived DTOs and OpenAPI, verified in this repo's e2e suite.- A TTL-declared field is written as a
Timestampand the emitted artifact contains the
matching TTL policy.- Every query the e2e suite issues has a corresponding entry in the emitted index artifact;
a CI job asserts the suite runs clean against an emulator started with only those indexes.- An entity declaring an unsupported relation fails at boot with the relation named.
selectreduces the fields returned (asserted at the backend seam).
Phase 3 — performance, limits and confidence
P3-1 · Cursor pagination (matrix #13) — effort S in the adapter, M with the core option.
Add cursor/after to RepositoryFindOptions (opaque, adapter-encoded); Firestore maps it
to startAfter — the natural fit, since Firestore cursors are keyset. TypeORM maps it to
the (a, b) < (x, y) predicate apps write by hand today. Field evidence: the consumer app
hand-wrote keyset lists three times on SQL (bypass-log rows #17, #21, #25 in
realtystack/docs/benchmark/00-bypass-log.md) and its Firestore predecessor hand-wrote a
startAfter helper (see Prior art).
P3-2 · Read-cost observability — effort S. Surface document-read counts per operation
(debug log or a hook), so the difference between a pushed query and a client-side scan is
visible in tests. Firestore bills per read; a silent scan is a production incident, not a
slow test.
P3-3 · Enforce Firestore semantics in the test double (matrix #22) — effort M.
InMemoryFirestoreBackend should reject what Firestore rejects: missing composite index
(against a declared index set), inequality-without-leading-orderBy, in > 30, batch > 500,
read-after-write inside a transaction. Today it filters and sorts everything happily
(src/backends/in-memory-firestore.backend.ts:74-89), which is why these defects are
invisible to CI.
P3-4 · Emulator suite parity (criterion (e)) — effort M. Broaden
vitest.firestore.config.mts beyond *.emulator-spec.ts value-parity tests so the
repository/e2e suites that run on SQLite also run on the emulator in CI.
P3-5 · Documentation and correctness cleanups — effort S.
README.md:192-194claims document-idINaccepts "at most 500 ids" in "one Admin SDK
getAllrequest". The code enforces no cap
(src/repository/firestore-where.translator.ts:179-187) and chunks by 300 across
multiplegetAllcalls (src/backends/admin-firestore.backend.ts:24,164-184). Fix the doc
or enforce the cap.WhereOperator.CONTAINSwith an array value maps toarray-containswith the whole
array as the comparison value (src/repository/firestore-where.translator.ts:263-273);
Firestore'sarray-containsmatches a single element, andarray-contains-anyis the
list operator. No test covers this path (grep "array-contains" src/__tests__→ 0 hits).- Subcollections /
collectionGroupare unsupported; every entity is a flat top-level
collection (src/repository/firestore-repository.ts:45-49). Sub-resources therefore model
as flat collections with an FK field. Worth an explicit README section, and optionally a
parent/subcollectionregistration option (effort L).
Phase 3 acceptance criteria
- A generated
listop returns a cursor page compiled tostartAfter, proven against the
emulator.- Read-count assertions exist for the list/count paths and fail if a scan regresses.
- The in-memory backend rejects each of the five illegal query shapes above, with tests.
- CI runs the repository/e2e suite on the emulator, not only the value-parity file.
Assumptions corrected
Recorded so this issue does not misstate the package — each of these was on our own
"missing" list and is wrong:
count()is implemented, with the server-side aggregation
(src/backends/admin-firestore.backend.ts:150-152). The caveat is P1-4, not absence.- Soft delete and restore are implemented (
src/repository/firestore-repository.ts:167-190),
includingwithDeleted— with the detection caveat in P2-1. skip/takeare pushed to the server on the clean path
(admin-firestore.backend.ts:123-131), not only sliced in memory. The problem is how
rarely that path is reachable.IN ≤ 30is enforced in code with a clear error
(src/repository/firestore-where.translator.ts:224-234). The "500" document-id limit,
however, exists only in the README — see P3-5.- OR is supported via
RepositoryAdapter.toDnf()with client-side branch merging
(src/repository/firestore-query-runner.ts:20-53), and the "one inequality field per
query" rule is enforced with a good error
(src/repository/firestore-where.translator.ts:315-331). - The transaction extension point already exists in the contract, so P1-1 is additive for
rockets-core; onlyFirestoreBackendtakes a breaking change. - Cursor pagination is partly a core gap, not only a Firestore gap —
RepositoryFindOptionshas no cursor field for any adapter, TypeORM included. - Schema migrations are not a Firestore gap — there is nothing to migrate; the real
deliverable is index/TTL artifact emission.
Prior art (field evidence)
A full Firestore POC on Rockets (consumer repo, branch feat/rockets-migration) ran the
same application on the raw Firebase SDK behind Rockets and ended with 15 explicitly marked
bypass sites in 8 files. Reproduce with git grep -n "rockets-gap:" feat/rockets-migration:
- Transactions — 9 sites (the figure quoted in the consumer's recon notes):
api/src/chat/chat-session.service.ts:111(deleteSession),:122(reserveTurn),:182
(completeTurn),:216(clearTurn);api/src/reports/cma/cma-share.service.ts:92
(createOwnerShare),:123(revokeOwnerShare);api/src/video/video.service.ts:38
(enqueueVideoGeneration),:101(reserveIdempotentRecord);
api/src/video/pipeline/upload-video.ts:175(completeOwnedRenderLease).
(RawrunTransactioncalls in that POC'ssrcnumber 10 — the rate-limit one is filed
underttl.) - Cursor pagination — 4 sites:
api/src/common/firestore/list-owned-artifacts.ts:29(the
shared keyset helper — it callsstartAfter(snapshot)at:88, the exact primitive the
adapter does not expose),api/src/chat/chat-session.service.ts:85,
api/src/reports/cma/cma-pdf.service.ts:180,api/src/video/video.service.ts:295. - TTL — 2 sites:
api/src/common/rate-limit/rate-limit.service.ts:64,
api/src/common/cache/analysis-response-cache.service.ts:100.
Consumer gap register (realtystack/docs/benchmark/00-gap-register.md) — rows that bear
directly on this issue:
- #1 (line 7) — no Firestore zod compiler; the reason the benchmark chose TypeORM → P2-1.
- #2 (line 8) — the Firestore→SQL swap and its contract-invisible divergences (deep-health
firestore=1→database=1,rateLimitStore: "database") → the whole issue. - #43 (line 54) — the CRUD list grammar cannot carry a computed resource's parameters
(1 of 17 map). Adapter-neutral, but it is the precedent for "the grammar is a language of
predicates over a result set" — relevant when specifying what P3-1's cursor should be. - #45 (line 56) — a schema with no persistence metadata is invisible to the DTO layer and
array columns ship empty → the warning in P2-1. - #51 (line 62) — a repo call made from inside a hook must forward
{ ctx }or it commits
outside the operation's transaction and runs with hooks off. On Firestore todayctxis
discarded unconditionally, so the first half of that footgun is permanent rather than
opt-in → P1-1.
Related bypass-log rows (realtystack/docs/benchmark/00-bypass-log.md): #17/#21/#25 (three
hand-written keyset lists → P3-1), #26 (idempotent transactional enqueue → P1-1/P1-2), #27
(worker lease state machine → P1-1/P1-2), #4/#10/#20 (transactional rate-limit store →
P1-1/P1-2/P1-3).
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from btwld/rockets
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
question
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
question
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
bug
Difficulty 5/5 Over a week Newbie friendliness 45/100
Similar issues
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 92/100
lingdojo/kana-dojo#34449 · 1 comment · 5 reactions ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
drizzle-team/drizzle-orm#6726 ·
Maintainers usually reply within 2 days
-
bug platform-desktop platform-windows
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Termix-SSH/Termix#1951 · 2 comments ·
Maintainers usually reply within 1 day
-
Grok provider voice list shows 5 of xAI's 28 built-in voicesPossibly taken @hkjarral claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
hkjarral/AVA-AI-Voice-Agent-for-Asterisk#883 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100