Vintage-Aware “As-Of” Analytics + Release Radar + Revision Heatmaps (ALFRED-backed)

Summary

Add vintage-aware pipelines and UI so users can:

1. view each series as it was published on a past date (ALFRED vintages),
2. see revision heatmaps and “surprise” deltas by release,
3. get a Release Radar calendar with alerts, and
4. ask the chatbot counterfactual questions (“Given data as of 2022-06-01, what was the GDP signal?”) with as-of constrained RAG.

Why it matters

• Most macro workflows hinge on data revisions and publication lags.
• Vintage context reduces hindsight bias and improves model backtests & narratives.
• Differentiates this app from generic FRED viewers and plain RAG chat.

---

Scope (v1)

• Ingestion: Pull vintages for selected tickers (CPI, PCE, GDP, UNRATE, DGS10, INDPRO, HOUST, M2, SP500, VIX, etc.). Store both current and vintage snapshots.
• API: /series/:id?asOf=YYYY-MM-DD returns the value as published by that date.
• UI: Toggle “As-Of Mode” on charts; revision heatmap (release on x-axis, vintaged period on y-axis, color = revision).
• Radar: Release calendar + optional email/webhook alerts; show last vs. prior, and rolling surprise vs. 6-month average.
• RAG: Chat constraints to as-of vector slices so answers reflect only what was known then.

---

Data model (Mongo)

SeriesMeta { _id, fredId, title, frequency, units, seasonal, updatedAt }

SeriesPoint {
  _id, fredId, date, value, // latest-publication value
  vintages: [{ asOf: Date, value: number }] // append-only
}

ReleaseCalendar {
  _id, fredId, releaseName, releaseDates: Date[], tz: string
}

RevisionStat {
  _id, fredId, period: Date,
  firstValue: number, latestValue: number,
  maxAbsRev: number, meanAbsRev: number, lastUpdated: Date
}

---

API (Express)

• GET /api/series/:fredId?asOf=YYYY-MM-DD → time series truncated to vintage values.
• GET /api/series/:fredId/revisions?window=5y → revision table & summary stats.
• GET /api/releases/upcoming?days=30 → Release Radar feed.
• POST /api/alerts → create alert (series, threshold or “on release”).
• POST /api/chat body { query, asOf?: "YYYY-MM-DD" } → RAG uses vintage slice when asOf provided.

---

Frontend (React/MUI)

• Chart page: As-Of switch + date picker; badge showing “vintage snapshot”.
• Revision Heatmap component with tooltips (orig vs. latest, Δ).
• Release Radar page: calendar list, subscribe to alerts; mini “surprise card”.
• Chat: “Use As-Of” toggle and a chip displaying the active cutoff.

---

RAG changes

• Index two namespaces in Pinecone: current and vintage:<YYYY-MM-DD> (or a compact bucketing by month).
• If asOf present, route queries to nearest vintage namespace and mask newer docs.
• System prompt note: “Only use data with publication date ≤ asOf.”

---

Acceptance criteria

• For a series with known revisions (e.g., GDP), chart matches vintage values when an as-of date is chosen.
• Heatmap renders with correct axes and deltas; hovering shows first vs. latest.
• Chat with asOf=2020-06-01 does not cite data published after that date.
• Release Radar lists next 30 days, with at least 5 major tickers populated.
• Basic alert works (store record + mock webhook/email log).

---

Tasks

Backend

• ALFRED/FRED vintage fetcher + pagination; backfill + nightly cron.
• Upsert strategy: append vintages array; dedupe on (fredId, date, asOf).
• Revision stats job (mean/max absolute revision by period).
• Release calendar seeds + updater.
• Pinecone: create current + vintage-YYYYMM namespaces; writer util.
• asOf router + validation; unit tests.

Frontend

• As-Of toggle/date picker; vintage warning chip.
• Revision Heatmap (MUI + Recharts): virtualization for long histories.
• Release Radar page + alert modal.
• Chat “As-Of” mode UI; display provenance (“data up to …”).
• Empty/error states + skeletons.

DevOps

• .env: VINTAGE_CRON, VINTAGE_NAMESPACE_RETENTION, ALERTS_WEBHOOK_BASE.
• Seed scripts & docs; add Swagger for new endpoints.

---

Stretch (future issues)

• Nowcasting with mixed-frequency data using only as-of inputs.
• Model drift dashboard: how revisions change backtest accuracy.
• WebSub-style push for release alerts.