Empty insider FileDate fallback loses the known filing calendar day

Open
#21 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
49/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp
Domain
data

Research direction

Read DataProcessing/QuiverInsiderTradingDataDownloader.cs around lines 119–124 and the QuiverInsiderTrading.Reader and QuiverInsiderTradingUniverse.Reader implementations. Trace how each reader interprets an empty FileDate, then inspect the existing NUnit coverage and add cases for omitted same-day timestamps and genuinely unknown dates. Done means both readers preserve the known upload calendar day while the stated timestamp, transaction date, and legacy-column behavior remains covered.

Written by the indexing model from the issue text.

Description

priority:P2

Problem

The insider-trading downloader deliberately omits FileDate when its calendar day equals Uploaded, but both readers reconstruct an empty value as the upload day minus one day. Consequently, a known filing day is changed, not merely its intraday precision discarded.

This is already present on master at 7ff2155; it is separate from the explicit-timestamp correction in #17 / #19.

Evidence

  • DataProcessing/QuiverInsiderTradingDataDownloader.cs:119–124 writes an empty field when insiderTrade.FileDate.Date == uploadedDate. Its comment says this preserves the calendar day while dropping intraday precision.
  • QuiverInsiderTrading.Reader uses uploadedDate.AddDays(-1) for that empty field.
  • QuiverInsiderTradingUniverse.Reader likewise uses date.AddDays(-1); universe files are keyed by upload date.
  • Both classes document FileDate as the time the transaction was filed and became publicly available.

For example, a filing at 2026-05-08 09:30:00 uploaded on 2026-05-08 is written with an empty filing-date field. The resulting rows have the following shape:

20260508,,20260507,P,150.25,100,500,A,D,Example Person,CEO,T,T,F,
AAPL R735QTJ8XC9X,AAPL,,20260507,P,150.25,100,500,A,D,Example Person,CEO,T,T,F,

Parse the first row with an AAPL subscription configuration and the second with the universe reader, using read date 2026-05-08.

Two local NUnit assertions against the base readers reproduce:

Expected FileDate: 2026-05-08 00:00:00
Actual FileDate:   2026-05-07 00:00:00
Failed: 2, Passed: 0, Total: 2

Required outcome

For a filing timestamp omitted because its day matches the upload day, both readers must preserve that known calendar day. Keep Time/EndTime, transaction dates, explicit filing timestamps, and legacy-column support unchanged. Add regression coverage for the downloader/reader contract, including how genuinely unknown filing dates should be represented; the same empty field currently covers both cases.

Suggestion: align the missing-value reconstruction with the writer's omission rule rather than changing explicitly stored timestamps again. Recovering the discarded intraday precision is a separate concern.

Priority P2: the verified impact is limited to filing-date metadata; this does not change the availability timestamps.

Dominant language
C#
Stars
0
Forks
3
Avg merge
39m
Merged PRs (30d)
4

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from QuantConnect/Lean.DataSource.QuiverQuant

All issues in QuantConnect/Lean.DataSource.QuiverQuant

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.