QuiverInsiderTrading readers throw (fatal) on rows in a legacy column layout; validate column count and return null

Start with QuiverInsiderTradingUniverse.cs:150 and QuiverInsiderTrading.cs, then review the cited SubscriptionDataReader.cs handling of BaseData.Reader returning null. Reproduce the malformed rows over the 2026-07-27 to 2026-08-05 window; done means unexpected column counts or unparseable fields are skipped without a reader exception, while the current layouts still produce data points.

Summary

Both QuiverInsiderTradingUniverse.Reader and QuiverInsiderTrading.Reader assume every CSV line is in the current 16-column (universe) / 15-column (per-ticker) layout. A line in any other shape throws out of the reader, and LEAN treats a reader exception as a fatal runtime error (SubscriptionDataReader.cs:551 -> ReaderErrorDetected -> IResultHandler.RuntimeError), so one stray row in one daily file stops every algorithm that subscribes to the universe. The readers should validate the column count and return null for a line they cannot parse, so a bad row costs one data point instead of the whole backtest.

This is a sibling of #9 (legacy 14-column rows after the Name field was added). The rows here are older still: the pre-a50ad5f1 layout, and they fail on the date parse before any index is out of range, so the hasName fallback proposed in #9 does not cover them.

Evidence

Universe file alternative/quiver/insidertrading/universe/20260731.csv on the cloud data store (read from a backtest on LEAN 2.5.0.0.18114, 2026-09-21): 663 rows, 497 in the 16-column layout and 166 in the pre-refactor 6-column layout {sid},{ticker},{name},{shares},{price},{sharesOwnedFollowing}, spanning 37 tickers. Example:

ASPS UF02B1SL6QG5,ASPS,winkler matthew t.,1000.0,5.045,39989.0

The matching per-ticker file asps.csv carries the same trades twice: once in the 15-column layout and once as 5-column {date},{name},{shares},{price},{sharesOwnedFollowing} rows:

20260731,,20260730,P,5.045,1000.0,39989.0,A,D,Winkler Matthew T.,,T,F,F,F
20260731,winkler matthew t.,1000.0,5.045,39989.0

Every other universe file from 2026-05-01 to 2026-09-19 (99 files) is 16-column only, so the stray rows come from a single run of a pre-refactor build (the redundancy job for this dataset is still pinned to 6d71e402, which predates a50ad5f1) for upload date 2026-07-31; ProcessUniverse() then copied them into the universe file because it keys rows on the first column only (QuiverInsiderTradingDataDownloader.cs:219-251).

Reader failure, QuiverInsiderTradingUniverse.cs:150: csv[2] is winkler matthew t. on a 6-column row, and Parse.DateTimeExact(s, "yyyyMMddHHmmss") throws:

Error invoking QUIVERINSIDERTRADINGUNIVERSE-USA-... .QuiverInsiderTradingUniverse 2S data reader. Line: ASPS UF02B1SL6QG5,ASPS,winkler matthew t.,1000.0,5.045,39989.0 Error: String 'winkler matthew t.' was not recognized as a valid DateTime.

Same for the per-ticker reader, QuiverInsiderTrading.cs (csv[1] parsed as yyyyMMddHHmmss):

Error invoking ASPS.QuiverInsiderTrading data reader. Line: 20260731,winkler matthew t.,701.0,5.1,38989.0 Error: String 'winkler matthew t.' was not recognized as a valid DateTime.

Reproduced in a cloud backtest with AddUniverse<QuiverInsiderTradingUniverse>(_ => Enumerable.Empty<Symbol>()) over 2026-07-27 to 2026-08-05, and with AddData<QuiverInsiderTrading>("ASPS") over the same window. Note that the read-ahead worker (SubscriptionUtils.CreateAndScheduleWorker) stamps the runtime error with the algorithm clock, so a reporter sees the failure "on" an earlier date than the file that holds the row (here 2026-07-07, because there are no universe files between 2026-07-09 and 2026-07-26).

Proposed change

1. In both readers, check csv.Length against the expected column count first and return null (optionally Log.Error once per file) when it does not match, instead of letting the parse throw. BaseData.Reader returning null is already handled by TextSubscriptionDataSourceReader as "no data point". Combined with the #9 fallback this makes the readers tolerant of every layout the store has carried.
2. Wrap the Parse.DateTimeExact / decimal.Parse calls so a single unparseable field also yields null rather than an exception.
3. Separately (data side, not this repo): strip the 5-column rows from the 37 per-ticker files and let ProcessUniverse() regenerate 20260731.csv; repoint the redundancy job at the current commit so a pre-refactor build cannot write again.

Open questions

• Should ProcessUniverse() itself skip lines whose column count does not match the current layout, so a per-ticker file with stray rows cannot poison the universe file? It already skips lines whose first column is not a yyyyMMdd date.

Reference

Intercom conversation: 215476021685114.