Add a pickle-size regression test for fitted learners (v1.0 M1)
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 67/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning
Research direction
Read the v1.0 M1 roadmap item in #938 and inspect the small datasets in conftest. Add one test file covering fitted meta-learners, causal and uplift trees, and forests by pickling each model and checking its size against the proposed budget of about twice today's size. Done means the test passes in the normal suite and catches pickle sizes that grow with training rows.
Written by the indexing model from the issue text.
Description
The v1.0 roadmap in #938 lists "a pickle-size regression guard in CI" under M1. I'd like to pick this up.
The plan is one test file that fits each meta-learner (S, T, X, R, DR, regressor and classifier) plus the causal and uplift tree and forest on the small conftest datasets, pickles each one, and checks the size against a budget of about 2x today's size. Today these pickle to roughly 1 to 20 KB and don't grow with the number of training rows, so a change that starts keeping training data on a fitted model would fail the test right away. It runs in the normal test suite in about 10 seconds with no CI changes.
Happy to adjust the learner list or headroom if you have a preference.
- Dominant language
- Python
- Stars
- 6k
- Forks
- 892
- Avg merge
- 11h 53m
- Merged PRs (30d)
- 1
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from uber/causalml
-
BaseRClassifier ignores cv_n_jobs during outcome cross-validationPossibly taken @ayuxy027 claimed this 17 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
-
cate_scoring default learner is oversized for its own per-fold, per-arm call patternPossibly taken @su-jin1425 claimed this 78 days ago. Openenhancement
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
S-, T- and X-learner bootstrap confidence intervals are not reproduciblePossibly taken @su-jin1425 claimed this 78 days ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
sklearn metadata routing: let `treatment` reach a learner inside a PipelineMay be free again @su-jin1425 claimed this 90 days ago, and no pull request is open. Openenhancement
Difficulty 4/5 3-5 days Newbie friendliness 55/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
pierreaubert/gpui-toolkit#114 ·
Maintainers usually reply within 1 day
-
intake mcp-intake needs-human-review priority:high type:bug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Ikalus1988/MisakaNet#4903 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
FinanceFlash/unvibecode#819 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day