pytorch/serve

Accelerated preprocessing

Open

Aperta il 1 apr 2022

Vedi su GitHub
 (1 commento) (3 reazioni) (0 assegnatari)Java (3844 star) (790 fork)batch import
enhancementhelp wanted

Descrizione

This recent PR by @min-jean-cho https://github.com/pytorch/serve/pull/1545 has made me realize how much performance we're leaving on the table when it comes to acceleration of preprocessing operations. We could more easily support more torchvision backends, faster tokenizers and add more hardware specific configurations to make it for developers to pick and choose.

This will also help our #1457 efforts since there we'd only considered so far performance of torchserve and inference()

  • Accelerated decoding
  • Accelerated preprocessing ops
  • Graph mode accelerated processing ops - like DALI

Guida contributor

Tech stack
python
Dominio
performancemachine learning
Tipo issue
feature
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
3
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
1-2 days
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
stale
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
mostly clear
Prerequisiti
Basic knowledge of PyTorch ServePython programmingFamiliarity with torchvision
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
40
Direzione di ricerca
Review the linked PR #1545 for acceleration ideas. Investigate the current preprocessing pipeline in PyTorch Serve, focusing on image and text preprocessing. Consider integrating torchvision's acceleration features and hardware specific optimizations. Examine existing code in the `serve/` directory for potential entry points to implement accelerated decoding and preprocessing ops.