Faster image pipelines · FluxML/FastAI.jl#235

(0 留言) (0 反應) (0 負責人)Julia (592 star) (49 fork)batch import

enhancementhelp wanted

描述

Some low-hanging fruit that FastAI.jl can pick to become faster here:

builtin presizing support
- prototype at https://github.com/lorenzoh/DataBlockStorage.jl/
- using JpegTurbo.jl's downsized loading so no storing to disk is needed (https://github.com/JuliaIO/JpegTurbo.jl/issues/15#issuecomment-1031548344)
use inplace operations
- DataAugmentation.jl already supports inplace operations, FastAI.jl just needs to add a buffered encode! interface and use those paths
- JpegTurbo.jl image loading could also be done inplace (https://github.com/JuliaIO/JpegTurbo.jl/issues/23)
- the whole pipeline could become non-allocating
use MLUtils.jl (#196)
- new DataLoader seems more stable and possibly a little faster
run projective transformations on GPU (https://github.com/lorenzoh/DataAugmentation.jl/issues/48)

技術棧: 無
領域: performancemachine learning
議題類型: performance
難度: 4
預計時間: over 1 week
活動狀態: stale
清晰度: clear
前置要求: Julia programmingImage processingFastAI.jl basicsJpegTurbo
新手友善度: 20
研究方向: The issue outlines several performance improvements. Start by examining the prototype in DataBlockStorage.jl and the JpegTurbo.jl issue for downsized loading. Then look into implementing a buffered `encode!` interface for inplace operations. Also review MLUtils.jl integration and GPU support in DataAugmentation.jl. The repository is FastAI.jl; check the source code for image pipelines.