rosejn/cortex
在 GitHub 查看Memory usage issues (OOM exceptions) with classification confusion matrix design
Open
#101 建立於 2017年2月19日
bugharderhelp wanted
描述
We use far too much memory during classification and this actually has nothing to do with cortex proper but with the display of the confusion matrix.
The issue is that the confusion matrix stores images and not paths or something like that. Thus it ends up with the entire cross-validation dataset held in memory which is untenable for larger datasets.
In general I think that it is unnecessary to display the entire dataset for the confusion matrix or at least it isn't necessary to show more than like 10 or 20 examples per entry.
In any case, this causes current projects to sometimes run into OOM exceptions so setting up some testing environment for this and working with it a bit to minimize this would help.