pytorch/serve

Having two different endpoints for a single model

Open

#2,058 创建于 2023年1月5日

在 GitHub 查看
 (2 评论) (0 反应) (0 负责人)Java (790 fork)batch import
enhancementhelp wantedtriaged

仓库指标

Star
 (3,844 star)
PR 合并指标
 (30 天内没有已合并 PR)

描述

Hello!

Is it possible to create two different endpoints for a single model? So it can work in two separate modes and batch accordingly depending on the input (e.g., it can generate an image or perform upscaling given an input). I can make two different models but then the memory will be double, while actually no extra memory is needed (as the model weights are the same). Ideally, I want to have the same model weights but two different endpoints with different batching settings.

Best, Valentin

贡献者指南