Faster image pipelines · FluxML/FastAI.jl

(0 commentaire) (0 réaction) (0 personne assignée)Julia (49 forks)batch import

enhancementhelp wanted

Métriques du dépôt

Some low-hanging fruit that FastAI.jl can pick to become faster here:

builtin presizing support
- prototype at https://github.com/lorenzoh/DataBlockStorage.jl/
- using JpegTurbo.jl's downsized loading so no storing to disk is needed (https://github.com/JuliaIO/JpegTurbo.jl/issues/15#issuecomment-1031548344)
use inplace operations
- DataAugmentation.jl already supports inplace operations, FastAI.jl just needs to add a buffered encode! interface and use those paths
- JpegTurbo.jl image loading could also be done inplace (https://github.com/JuliaIO/JpegTurbo.jl/issues/23)
- the whole pipeline could become non-allocating
use MLUtils.jl (#196)
- new DataLoader seems more stable and possibly a little faster
run projective transformations on GPU (https://github.com/lorenzoh/DataAugmentation.jl/issues/48)

Direction de recherche: Concentrez vous sur la mise en œuvre du support de preredimensionnement en utilisant le chargement reduit de JpegTurbo.jl, en suivant le prototype dans DataBlockStorage.jl. Testez l'integration et evaluez les ameliorations de vitesse.
Stack technique: julia
Domaine: machine learning
Type d'issue: Performance
Difficulté: 2
Temps estimé: Une demi journée
Statut d'activité: Récente
Clarté: Claire
Prérequis: JuliaGitFastAI.jl
Accessibilité débutant: 70