lm-sys/FastChat

TensorRT-LLM backend support?

Open

#2.595 aberto em 21 de out. de 2023

Ver no GitHub
 (3 comments) (7 reactions) (0 assignees)Python (38.959 stars) (4.736 forks)batch import
enhancementgood first issue

Description

Any plan to NVIDIA's new tensorrt llm backend? https://github.com/NVIDIA/TensorRT-LLM/tree/release/0.5.0

Guia do colaborador