rosejn/cortex

Memory usage issues (OOM exceptions) with classification confusion matrix design

Open

#101 opened on Feb 19, 2017

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Clojure (1,269 stars) (109 forks)batch import
bugharderhelp wanted

Description

We use far too much memory during classification and this actually has nothing to do with cortex proper but with the display of the confusion matrix.

The issue is that the confusion matrix stores images and not paths or something like that. Thus it ends up with the entire cross-validation dataset held in memory which is untenable for larger datasets.

In general I think that it is unnecessary to display the entire dataset for the confusion matrix or at least it isn't necessary to show more than like 10 or 20 examples per entry.

In any case, this causes current projects to sometimes run into OOM exceptions so setting up some testing environment for this and working with it a bit to minimize this would help.

Contributor guide

Tech stack
None
Domain
machine learningperformance
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Basic understanding of ClojureFamiliarity with machine learning confusion matrices
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
Investigate the confusion matrix implementation in the cortex project. Identify where images are stored instead of paths. Consider limiting the number of examples shown per entry to reduce memory usage. Set up a test with a large dataset to reproduce OOM exceptions and verify the fix.