[Bug]: `trtllm-serve` renders a client-supplied `chat_template` server-side, a string-blow-up expression OOM-kills whole server (unauthenticated crash DoS)

Ouverte
#4,859 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 6 jours

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
48/100
Type d'issue
Bug
Clarté
Plutôt claire
Activité
Active
Stack technique
python
Domaine
api, backend, security

Piste de recherche

Start with the reproduction script, then inspect tensorrt_llm/serve/openai_server.py around lines 561-562 and ChatCompletionRequest in openai_protocol.py around line 622. Run the provided reproduction against trtllm-serve to confirm the current behavior. Done means client-supplied templates cannot exhaust server memory or take the service down, while normal chat completions continue to work.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

/v1/chat/completions accepts a per-request chat_template (a Jinja template string) that is compiled and rendered server-side. A remote, unauthenticated client therefore controls a template the server executes. The Jinja environment blocks a runaway range() loop (returns 400), but it does not block a string-multiplication expression, so a single request with

{"model":"m","messages":[{"role":"user","content":"hi"}],"chat_template":"{{ \"a\" * 1000000000 }}"}

makes the server allocate a multi-gigabyte string and the process is OOM-killed (observed: Detected 7 oom_kill events … OOM Killed; /health stops answering). One request takes the whole server down for all clients.

That the client template is truly evaluated server-side is confirmed independently: a chat_template of {{ 1/0 }} returns a 400 whose message leaks a ZeroDivisionError from the template. (This also means there is a server-side template-injection surface; HF's apply_chat_template uses a sandboxed Jinja environment, so escalation to RCE is likely blocked, but the memory-exhaustion DoS is not.)

Cause

tensorrt_llm/serve/openai_server.py (~lines 561-562) passes the request field straight into the renderer:

apply_chat_template(..., chat_template=request.chat_template or self.chat_template, chat_template_kwargs=request.chat_template_kwargs or {})

with ChatCompletionRequest.chat_template: Optional[str] (openai_protocol.py ~622). There is no restriction on accepting a client template, and the sandbox does not bound the memory/compute a template may consume (string multiplication, and likely other allocation-heavy expressions, are not limited).

Reproduction

Use repro_trtllm_chat_template_oom.py

trtllm-serve Qwen/Qwen2.5-0.5B-Instruct --host 0.0.0.0 --port 8000 python3 repro_trtllm_chat_template_oom.py --base-url http://localhost:8000 --model-id Qwen/Qwen2.5-0.5B-Instruct
Langage dominant
C++
Étoiles
13.5k
Forks
2.4k
Merge moyen
6 j 21 h
PR mergées (30 j)
3

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de NVIDIA/TensorRT

Toutes les issues de NVIDIA/TensorRT

Issues similaires

Plus d'issues C++

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.