pytorch/serve

Having two different endpoints for a single model

Open

#2,058 建立於 2023年1月5日

在 GitHub 查看
 (2 留言) (0 反應) (0 負責人)Java (790 fork)batch import
enhancementhelp wantedtriaged

倉庫指標

Star
 (3,844 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

Hello!

Is it possible to create two different endpoints for a single model? So it can work in two separate modes and batch accordingly depending on the input (e.g., it can generate an image or perform upscaling given an input). I can make two different models but then the memory will be double, while actually no extra memory is needed (as the model weights are the same). Ideally, I want to have the same model weights but two different endpoints with different batching settings.

Best, Valentin

貢獻者指南