dnewcome/mlprobe

View on GitHub

Framework-agnostic profiler for ML training: audit production-readiness, probe on small data subsets, extrapolate cost and quality to full-dataset training across SageMaker/EC2 targets.

Stars: 1
Forks: 1
Open beginner issues: 0
Indexed issues: 22

Dominant language: Python
License: MIT
Last GitHub push: May 29, 2026
Latest indexed: Sep 21, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

22 open issues indexed

Pairwise gold-standard evaluation for entity resolution / deduplication Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

dnewcome/mlprobe#27 · Opened May 28, 2026
MLflow multi-run comparison table Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

dnewcome/mlprobe#26 · Opened May 28, 2026
Cluster / deduplication quality profiler Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

dnewcome/mlprobe#25 · Opened May 28, 2026
SageMaker training job launcher Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

dnewcome/mlprobe#24 · 1 comment · Opened May 28, 2026
EvalMetric: support sub-group breakdowns for source-level quality gates Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

dnewcome/mlprobe#23 · Opened May 28, 2026
Subset fraction propagation guard: warn when actual data size doesn't change across probe sizes Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

dnewcome/mlprobe#22 · Opened May 28, 2026
Scenario: regression_guard — declared quality targets with pass/fail gating Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

dnewcome/mlprobe#19 · 1 comment · Opened May 27, 2026
MLflow integration: from_mlflow_run() helper, mlprof-results-to-mlflow logging Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 42/100

dnewcome/mlprobe#17 · Opened May 21, 2026
Document ml-baker ↔ sagebaker relationship; avoid accidental rebuilding Open

enhancement

Difficulty 1/5 1-3 hours Newbie friendliness 74/100

dnewcome/mlprobe#16 · Opened May 21, 2026
Model size measurement (artifact size on disk, optional param count) Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 65/100

dnewcome/mlprobe#15 · Opened May 21, 2026
Quality-regression guard across declared variants Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 45/100

dnewcome/mlprobe#14 · Opened May 21, 2026
Stage-level profiling: per-stage time, GPU util, and bottleneck audit findings Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 45/100

dnewcome/mlprobe#13 · Opened May 21, 2026
Spot-pricing cost model with checkpointing-aware interruption overhead Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

dnewcome/mlprobe#12 · Opened May 21, 2026
Full empirical verification of checkpointing & incremental (interrupt + reference compare) Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

dnewcome/mlprobe#11 · Opened May 21, 2026
Chained probe mode for incremental training (cost amortization + warm-start curves) Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

dnewcome/mlprobe#10 · Opened May 21, 2026
Passive verification of checkpointing & incremental training Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 68/100

dnewcome/mlprobe#9 · Opened May 21, 2026
Inference profiling: load time, latency, VRAM, throughput Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 42/100

dnewcome/mlprobe#8 · Opened May 20, 2026
Support loading ModelSpec from YAML Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 62/100

dnewcome/mlprobe#7 · Opened May 20, 2026
Data-sampling tricks for higher-accuracy small-N quality estimates Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 42/100

dnewcome/mlprobe#6 · Opened May 20, 2026
Repetition averaging in the report layer Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 64/100

dnewcome/mlprobe#4 · Opened May 20, 2026
Saturating power-law fit for quality extrapolation Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 64/100

dnewcome/mlprobe#3 · Opened May 20, 2026
AWS Pricing API integration for live cost estimates Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 48/100

dnewcome/mlprobe#2 · Opened May 20, 2026

dnewcome/mlprobe

Get new issues in your inbox