Bug: Rsquare(N=0) expanding path leaks inf/garbage on near-constant windows (rolling path is guarded)

Open Beginner friendly
#2,297 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
75/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
numpy, pandas, python

Research direction

Start in qlib/data/ops.py at Rsquare._load_internal and compare the guarded rolling branch with the unguarded N == 0 expanding branch. Check the expanding standard deviation behavior on the reproduction series. Done means near-constant expanding windows return NaN rather than inf or spurious finite values, without changing the rolling behavior.

Written by the indexing model from the issue text.

Description

Description

Rsquare in qlib/data/ops.py computes R² via a Cython kernel as
num / sqrt(var_x * var_y). For a near-constant window var_y ≈ 0, so
floating-point cancellation yields inf or a spurious finite value instead of
NaN (a degenerate 0/0 regression).

Rsquare._load_internal guards against this by masking windows whose std is ≈0
to NaN — but only on the rolling (N != 0) branch:

def _load_internal(self, instrument, start_index, end_index, *args):
    _series = self.feature.load(instrument, start_index, end_index, *args)
    if self.N == 0:
        series = pd.Series(expanding_rsquare(_series.values), index=_series.index)
        # <-- no guard here
    else:
        series = pd.Series(rolling_rsquare(_series.values, self.N), index=_series.index)
        series.loc[np.isclose(_series.rolling(self.N, min_periods=1).std(), 0, atol=2e-05)] = np.nan
    return series

The expanding (N == 0) branch is unguarded, so Rsquare($feature, 0) returns
inf/garbage on near-constant windows. Because ops.py sets
np.seterr(invalid="ignore"), no warning is emitted — the bad values silently
propagate into features (e.g. Alpha158/Alpha360) and downstream models.

Reproduction

Near-constant series [100, 100, 100, 100.000001, 100, 100]:

expanding_rsquare (N==0 path): [nan, nan, nan, inf, 0.01717987, inf]
rolling_rsquare(4) after mask: [nan, nan, nan, nan, nan, nan]

The expanding path leaks inf and a spurious 0.0172; the rolling path is
correctly NaN.

Fix

Apply the same std≈0 → NaN mask on the expanding branch (using expanding std).
Slope/Resi are unaffected — they divide by the x-variance (index 1..N),
which is always well-conditioned; only Rsquare divides by the y-variance.

PR incoming.

Dominant language
Python
Stars
48.7k
Forks
7.7k
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/qlib

All issues in microsoft/qlib

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.