Single-date universe download also caches the next trading day

Abierto
  • ¿Algo no está bien?
#10,085 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
72/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
csharp
Área
data

Línea de trabajo

Read Engine/DataFeeds/DownloaderDataProvider.cs around lines 175-176 and Common/DataUniverseDownloaderGetParameters.cs around line 81, then follow the universe branch in Fetch and UniverseExtensions.RunUniverseDownloader. Check existing tests for DownloaderDataProvider and add coverage for a single requested date and the completed-day cutoff; done means no next-day file is written and incomplete current-day data is not cached.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

Fetching a single dated universe file downloads and writes the following trading day's universe too. If that following day is still in progress, partial data can be retained as a completed, date-based universe file and is not downloaded again.

Reproduction and observations

Use DownloaderDataProvider(IDataDownloader) with a deterministic downloader returning an underlying AAPL daily TradeBar and one AAPL option-contract daily TradeBar for each requested trade date, and empty enumerations for quote/open-interest requests. Configure Data as the relative data folder and create Data/option/usa/universes/aapl first.

  • Fetching Data/option/usa/universes/aapl/20241112.csv issues eight requests: four for November 12 and four for November 13. It can write both dated files even though only November 12 was requested.
  • Repeating this with yesterday's universe while today is a trading day writes today's universe file as well. Fetching today's file afterward makes zero new downloader requests:
requested-date=2026-11-10; extra-date-file=True; extra-fetch-new-requests=0

The latter check was run during the November 11 US session. A provider returning the current day's available bars can therefore leave an incomplete universe cached. This reproduces using previously supported relative paths, independently of absolute-path parsing.

Cause
  • Engine/DataFeeds/DownloaderDataProvider.cs:175-176 supplies processingDate through processingDate.AddDays(1) for a single universe-file request.
  • Common/DataUniverseDownloaderGetParameters.cs:81 uses Time.EachTradeableDay, whose end date is inclusive (Common/Time.cs:547-555).
  • The universe branch does not use the completed-data cutoff computed earlier in Fetch.
  • UniverseExtensions.RunUniverseDownloader writes a separate file for every enumerated date.
  • DownloaderDataProvider.NeedToDownload does not refresh existing date-based files.
Required outcome

A single-file universe request must not generate an unsolicited next-day file or cache an incomplete trading day's data. Cover both the requested date range and subsequent cache behavior. A suggestion is to pass the single processing date as both ends of the inclusive universe range, and enforce the completed-day cutoff for universe files too.

Lenguaje dominante
C#
Estrellas
22.5k
Forks
5.4k
Merge medio
1 d 2 min
PR fusionados (30 d)
71

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de QuantConnect/Lean

Todos los issues de QuantConnect/Lean

Issues similares

Más issues de C#

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.