help wantedonnxquestion
倉庫指標
- 星標
- (6,593 顆星)
- PR 合併指標
- (平均合併 2天 16小時) (30 天內合併 34 個 PR)
描述
The question is how do you free memory
https://github.com/triton-inference-server/onnxruntime_backend/issues/103
When the model is deployed to a single card, I can specify real-time release of gpu memory, but if the model is deployed to multiple cards, I don't know what the format looks like
parameters { key: "memory.enable_memory_arena_shrinkage" value: { string_value: "gpu:3" } }
instance_group [
{
count: 1
kind: KIND_GPU
gpus: [ 3 ]
}
]