help wantedonnxquestion
仓库指标
- Star
- (6,593 star)
- PR 合并指标
- (平均合并 2天 16小时) (30 天内合并 34 个 PR)
描述
The question is how do you free memory
https://github.com/triton-inference-server/onnxruntime_backend/issues/103
When the model is deployed to a single card, I can specify real-time release of gpu memory, but if the model is deployed to multiple cards, I don't know what the format looks like
parameters { key: "memory.enable_memory_arena_shrinkage" value: { string_value: "gpu:3" } }
instance_group [
{
count: 1
kind: KIND_GPU
gpus: [ 3 ]
}
]