rosejn/cortex

Memory usage issues (OOM exceptions) with classification confusion matrix design

Open

#101 创建于 2017年2月19日

在 GitHub 查看
 (2 评论) (0 反应) (0 负责人)Clojure (1,269 star) (109 fork)batch import
bugharderhelp wanted

描述

We use far too much memory during classification and this actually has nothing to do with cortex proper but with the display of the confusion matrix.

The issue is that the confusion matrix stores images and not paths or something like that. Thus it ends up with the entire cross-validation dataset held in memory which is untenable for larger datasets.

In general I think that it is unnecessary to display the entire dataset for the confusion matrix or at least it isn't necessary to show more than like 10 or 20 examples per entry.

In any case, this causes current projects to sometimes run into OOM exceptions so setting up some testing environment for this and working with it a bit to minimize this would help.

贡献者指南