Single-date universe download also caches the next trading day

Aperta
  • Qualcosa non va?
#10,085 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
72/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
csharp
Ambito
data

Direzione di ricerca

Read Engine/DataFeeds/DownloaderDataProvider.cs around lines 175-176 and Common/DataUniverseDownloaderGetParameters.cs around line 81, then follow the universe branch in Fetch and UniverseExtensions.RunUniverseDownloader. Check existing tests for DownloaderDataProvider and add coverage for a single requested date and the completed-day cutoff; done means no next-day file is written and incomplete current-day data is not cached.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

Fetching a single dated universe file downloads and writes the following trading day's universe too. If that following day is still in progress, partial data can be retained as a completed, date-based universe file and is not downloaded again.

Reproduction and observations

Use DownloaderDataProvider(IDataDownloader) with a deterministic downloader returning an underlying AAPL daily TradeBar and one AAPL option-contract daily TradeBar for each requested trade date, and empty enumerations for quote/open-interest requests. Configure Data as the relative data folder and create Data/option/usa/universes/aapl first.

  • Fetching Data/option/usa/universes/aapl/20241112.csv issues eight requests: four for November 12 and four for November 13. It can write both dated files even though only November 12 was requested.
  • Repeating this with yesterday's universe while today is a trading day writes today's universe file as well. Fetching today's file afterward makes zero new downloader requests:
requested-date=2026-11-10; extra-date-file=True; extra-fetch-new-requests=0

The latter check was run during the November 11 US session. A provider returning the current day's available bars can therefore leave an incomplete universe cached. This reproduces using previously supported relative paths, independently of absolute-path parsing.

Cause
  • Engine/DataFeeds/DownloaderDataProvider.cs:175-176 supplies processingDate through processingDate.AddDays(1) for a single universe-file request.
  • Common/DataUniverseDownloaderGetParameters.cs:81 uses Time.EachTradeableDay, whose end date is inclusive (Common/Time.cs:547-555).
  • The universe branch does not use the completed-data cutoff computed earlier in Fetch.
  • UniverseExtensions.RunUniverseDownloader writes a separate file for every enumerated date.
  • DownloaderDataProvider.NeedToDownload does not refresh existing date-based files.
Required outcome

A single-file universe request must not generate an unsolicited next-day file or cache an incomplete trading day's data. Cover both the requested date range and subsequent cache behavior. A suggestion is to pass the single processing date as both ends of the inclusive universe range, and enforce the completed-day cutoff for universe files too.

Lingua principale
C#
Stelle
22.5k
Fork
5.4k
Merge medio
22h 6m
PR unite (30g)
81

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di QuantConnect/Lean

Tutte le issue di QuantConnect/Lean

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.