Factor validation suite: port cross-sectional IC robustness tests (QuantGPT anti_overfit / adversarial_validator)
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 领域
- data, testing-qa
调研方向
Start by reading the statistical-testing skill and the two linked QuantGPT files, quantgpt/anti_overfit.py and quantgpt/adversarial_validator.py, to map the proposed tests and differences. The issue calls for a separate capability for cross-sectional factor panels, with deduplicated tests, translated scoring bands, configurable randomness, and faster permutation loops. Done means defining that capability and covering the listed tests; the issue does not name target files or tests in this repository.
由索引模型根据 Issue 内容生成。
描述
Context
While porting robustness-test ideas from QuantGPT (quantgpt/anti_overfit.py, quantgpt/adversarial_validator.py) into the statistical-testing skill, several tests could not be ported because they fundamentally require cross-sectional factor panel data (trade_date, stock_code, factor_value, daily_ret), not a single strategy return series (date + returns).
This issue tracks the unported remainder as a future factor-validation capability.
Unported tests
| Test | Why it doesn't port | What it needs |
|---|---|---|
IC half-life estimation (fit IC(t) = a·exp(-b·t) across forward periods 1/2/5/10/20/40d; pass: half-life > 5d) |
IC decay across horizons is a cross-sectional construct; no returns-series analog | Factor panel data; scipy.optimize.curve_fit |
Absolute IC thresholds (|IC| ≥ 0.02) |
Factor-magnitude-specific; meaningless for strategy returns | If ever ported, use effect-size-relative criteria instead of absolute thresholds |
| Label permutation / random universe / noise injection on factor values (full-strength versions) | Ported versions (sign-flip, block-bootstrap, parameter-plateau) are weaker analogs | Factor panel data |
Proposed design
- New standalone capability (separate skill or module), NOT folded into
statistical-testing(different data model, different routing intent). - Merge both QuantGPT files but dedupe: label permutation ≈ factor permutation (symmetric for Spearman IC) → ~7 distinct tests: IC stability, regime stress, permutation/placebo, time-shift decay, half-life, random universe, noise injection.
- Fixes needed vs upstream: translate Chinese strings, unify the two scoring bands (通过/基本通过/存疑/高风险 vs 推荐/谨慎/需改进/不推荐), parameterize
RandomState(42), vectorizegroupby().apply()permutation loops (slow on large panels). - Scoring bands (translated): ≥80 PASS / 60–79 CAUTION / 40–59 DOUBTFUL / <40 HIGH-RISK.
Sources
- 主要语言
- Python
- 星标
- 28
- 派生
- 17
- 平均合并
- 5 小时 7 分钟
- 30 天内合并 PR
- 1
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
uberdeveloper/fastbt 的其他 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 72/100
uberdeveloper/fastbt#29 · 2 条评论 ·
-
bug
难度 4/5 3-5 天 新手友好度 48/100
uberdeveloper/fastbt#31 ·
-
bug
难度 4/5 3-5 天 新手友好度 52/100
uberdeveloper/fastbt#30 ·
-
bug
难度 4/5 3-5 天 新手友好度 48/100
uberdeveloper/fastbt#28 ·
-
FB-008: close_trade mixes price and tick; stale exits are silent (zero-P&L fallback) [silent-wrong]未关闭bug
难度 3/5 1-2 天 新手友好度 70/100
uberdeveloper/fastbt#27 ·
查看 uberdeveloper/fastbt 的全部 Issue
相似的 Issue
-
python triage
难度 2/5 1-3 小时 新手友好度 82/100
microsoft/semantic-kernel#14590 ·
维护者通常 3 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 85/100
pinterest/pymemcache#662 ·
维护者通常 1 天内回复
-
难度 2/5 半天 新手友好度 62/100
gnocchixyz/gnocchi#1527 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 64/100
tenstorrent/tt-model-manager#181 ·
维护者通常 1 天内回复