Use of broadcasting data strucutures for daru internals.
#328 ouverte le 31 mars 2017
Métriques du dépôt
- Stars
- (1 060 stars)
- Métriques de merge PR
- (Métriques PR en attente)
Description
Currently, daru uses Arrays for storing data inside Vectors, which are collectively stored inside a dataframe.
However, this approach reduces the speed of most mathematical operations due to everything being a Ruby object and all the looping operations happening in Ruby.
I would like to explore alternatives to re-implementing daru's internal data structures in something like NMatrix or Numo::NArray for more efficient storage of data. @genya0407 's sake gem does this to some extent but it is still not as widespread as that of pandas.
This will most probably make use of broadcasting data structures. In the interest of speed, do you all think it would be alright to sacrifice compatibility with JRuby? Since NMatrix has a Java backend, how about implementing broadcasting in NMatrix and rewriting daru's internals using NMatrix?
Please pitch in your ideas into this thread.
CC: @mrkn @zverok @genya0407 @gnilrets @lokeshh @kozo2