[Bug]: `trtllm-serve` renders a client-supplied `chat_template` server-side, a string-blow-up expression OOM-kills whole server (unauthenticated crash DoS)

Abierto
#4,859 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 6 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
api, backend, security

Línea de trabajo

Start with the reproduction script, then inspect tensorrt_llm/serve/openai_server.py around lines 561-562 and ChatCompletionRequest in openai_protocol.py around line 622. Run the provided reproduction against trtllm-serve to confirm the current behavior. Done means client-supplied templates cannot exhaust server memory or take the service down, while normal chat completions continue to work.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

/v1/chat/completions accepts a per-request chat_template (a Jinja template string) that is compiled and rendered server-side. A remote, unauthenticated client therefore controls a template the server executes. The Jinja environment blocks a runaway range() loop (returns 400), but it does not block a string-multiplication expression, so a single request with

{"model":"m","messages":[{"role":"user","content":"hi"}],"chat_template":"{{ \"a\" * 1000000000 }}"}

makes the server allocate a multi-gigabyte string and the process is OOM-killed (observed: Detected 7 oom_kill events … OOM Killed; /health stops answering). One request takes the whole server down for all clients.

That the client template is truly evaluated server-side is confirmed independently: a chat_template of {{ 1/0 }} returns a 400 whose message leaks a ZeroDivisionError from the template. (This also means there is a server-side template-injection surface; HF's apply_chat_template uses a sandboxed Jinja environment, so escalation to RCE is likely blocked, but the memory-exhaustion DoS is not.)

Cause

tensorrt_llm/serve/openai_server.py (~lines 561-562) passes the request field straight into the renderer:

apply_chat_template(..., chat_template=request.chat_template or self.chat_template, chat_template_kwargs=request.chat_template_kwargs or {})

with ChatCompletionRequest.chat_template: Optional[str] (openai_protocol.py ~622). There is no restriction on accepting a client template, and the sandbox does not bound the memory/compute a template may consume (string multiplication, and likely other allocation-heavy expressions, are not limited).

Reproduction

Use repro_trtllm_chat_template_oom.py

trtllm-serve Qwen/Qwen2.5-0.5B-Instruct --host 0.0.0.0 --port 8000 python3 repro_trtllm_chat_template_oom.py --base-url http://localhost:8000 --model-id Qwen/Qwen2.5-0.5B-Instruct
Lenguaje dominante
C++
Estrellas
13.5k
Forks
2.4k
Merge medio
6 d 21 h
PR fusionados (30 d)
3

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/TensorRT

Todos los issues de NVIDIA/TensorRT

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.