Evaluate Model A and Model B on the test set
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 42/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start in scripts/evaluate_custom.py, which the issue names as the file to create or extend. Blockers #32, #23 and #25 must land first, since the saved checkpoints for Model A and Model B come from them. Compute accuracy, precision, recall, F1 and a confusion matrix per model, save figures/cm_model_A.png and figures/cm_model_B.png, and write results/metrics_custom.json. Done when that JSON contains entries for both models.
Written by the indexing model from the issue text.
Description
Load the saved checkpoints and compute all metrics.
Files: scripts/evaluate_custom.py
Tasks
- Accuracy, precision, recall, F1, confusion matrix for each model
- Confusion-matrix figures ->
figures/cm_model_A.png,figures/cm_model_B.png - Save metrics to
results/metrics_custom.json
Done when
Metrics JSON contains both models.
Depends on
#32 #23 #25
Close with a commit or PR message containing Closes #<this issue>.
- Dominant language
- Jupyter Notebook
- Stars
- 0
- Forks
- 0
- Avg merge
- 1m
- Merged PRs (30d)
- 3
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from VisionForgeCSE/CNN-For-Image-Classification
-
evaluation
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
-
evaluation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
evaluation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
training
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Optimizer factoryOpentraining
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
All issues in VisionForgeCSE/CNN-For-Image-Classification
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
SheffieldML/GPy#1217 ·
Maintainers usually reply within 1 day
-
area:docs
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
jejjohnson/geonnax#89 ·
Maintainers usually reply within 1 day
-
Triage 🩺
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
[Bug]: iter_prefetch crashes on accelerators without record_stream (MPS)Possibly taken @begcdn claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
vllm-project/llm-compressor#3269 ·
Maintainers usually reply within 1 day