tflearn/tflearn

Suggestions to re-structure 'data_*'

Open

#144 建立於 2016年6月17日

在 GitHub 查看
 (1 留言) (0 反應) (0 負責人)Python (2,425 fork)batch import
enhancementhelp wanted

倉庫指標

Star
 (9,602 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

Currently, there are many data modules:

  • data_utils.py: a set of utilities for numpy (to_categorical for labels, featurewise/samplewise transformations), for images (load_images, resize, etc...), for sequences (VocabularyProcessor, redundant seq from file, etc...) and for building datasets (build HDF5 dataset, image_preloader array ...)
  • data_flow.py: handles a DataFlow object responsible for creating the computation pipeline.
  • data_preprocessing: DataPreprocessing objects to be used with DataFlow.
  • data_augmentation: DataAugmentation objects to be used with DataFlow.

All of them are located under tflearn root path. For more clarity, we may organize them more efficiently, such as:

# Option 1: just moving them to a 'data' folder:
- tflearn/data
  - utils.py
  - builder.py (move here functions to create datasets)
  - preprocessing.py
  - augmentation.py
  - flow.py
# Option 2: same, but also splits utils into a new folder
- tflearn/data
  - utils/
     - image.py
     - numpy.py
     - sequence.py
  - builder.py (move here functions to create datasets)
  - preprocessing.py
  - augmentation.py
  - flow.py
# Option 3: Only re-organize utils functions (data_flow/aug/prep doesn't change, 
# but 'utils' and 'data_utils' are merged into a new same folder)
- tflearn/utils
     - core.py (from tflearn/utils.py) / or base.py / generic.py
     - image.py
     - numpy.py
     - sequence.py
     - dataset_builder.py (move here functions to create datasets)

Any suggestion and other ideas are more than welcome :)

貢獻者指南