Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

[Bug]: ollama-proxy: undocumented 120 s upstream response-header timeout turns a queued or cold-loading request into a 502

オープン
  • 何か問題がありますか？
#57 コメント 2 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 5 日以内に返信

@ckelseynv がすでに取り組んでいます。

2026年9月11日 から。

評価

この issue はまだ評価されていません。

説明

Summary

ollama-proxy returns 502 {"error":"upstream error: net/http: timeout awaiting response headers"} on any upstream request that has not produced response headers within 120 s. Ollama serves one request at a time by default (OLLAMA_NUM_PARALLEL=1) and sends no headers until generation starts, so a request that sits in Ollama's queue for more than 120 s, or that waits on a model load longer than 120 s, fails through PAIR while the identical request sent directly to Ollama succeeds. The limit is not mentioned in the README, services/ollama-proxy/README.md, the flag table, or docs/troubleshooting.mdx, and there is no flag or setting that changes it. 120 s is the observed value; I did not locate the constant in the source.

PAIR version or commit

v0.1.1 release archive service-binaries-linux-x64.zip: product 0.91.7, ollama-proxy 0.26.2, nvpair-ui-broker 0.40.2, run headless via nvpair-tui 0.7.2 in tmux with NVPAIR_LOG_LEVEL=debug.

Affected component

ollama-proxy

Environment
  • Node A: Ubuntu 24.04.4, kernel 6.8.0-138, RTX 3090 24 GB, Ollama 0.30.0
  • Node B: Ubuntu 24.04.4, RTX 3060 12 GB, Ollama 0.17.5
  • Two nodes paired. Ollama was pre-existing on both and adopted by PAIR on 11434; proxy on 11435.
Steps to reproduce

1. Queue depth (single node). Ollama on node A with qwen3.6:27b Q4_K_M (about 11 s per 200-token request on the 3090). Fire 20 independent POST /v1/chat/completions concurrently at the PAIR proxy (127.0.0.1:11435), max_tokens: 200, stream: false. Fire the same 20 at Ollama directly (127.0.0.1:11434) as the control.

Path Wall clock Succeeded Failed
Direct to Ollama 226.5 s 20 0
Through PAIR 120.3 s 10 10, all 502 at 120.24 to 120.31 s

Per-request completion times through PAIR: 12.7, 23.7, 35.2, 48.4, 58.5, 70.7, 81.5, 92.8, 104.5, 115.6 s, then ten failures at 120.2 to 120.3 s. Every request whose generation would have started after 120 s was dropped. GPU utilization on the node fell to zero within one second of the 502s: Ollama cancelled the orphaned queue entries when the proxy closed the connections.

2. Same test, 8 concurrent instead of 20, so the deepest queue position starts before 120 s:

Path Wall clock Succeeded Failed
Direct to Ollama 86.5 s 8 0
Through PAIR 85.6 s 8 0

Per-request completions through PAIR: 11.4, 22.4, 32.6, 42.6, 52.9, 64.4, 75.1, 85.6 s. No 502s. The only variable between this run and the 10-of-20 failure is queue depth.

3. Cold load (cluster, model only on the peer). Node B holds deepseek-r1:14b (9 GB) on a slow disk; cold load takes 2 to 3 minutes. From node A, POST /v1/chat/completions for that model to node A's proxy. PAIR routes it to node B correctly, then 502s at 120 s. Node B's Ollama abandoned the load when the connection dropped (/api/ps empty afterwards). A retry 30 s later succeeded in 31.6 s because the file was by then in page cache.

Expected behavior

Either no fixed header timeout on inference routes, or a default at least as long as Ollama's own OLLAMA_LOAD_TIMEOUT (5 minutes) with a way to configure it (flag on ollama-proxy, settings.json, or a per-engine setting), plus a line in the proxy README and troubleshooting doc stating the default and the error text it produces.

Actual behavior

A silent 120 s limit that fails requests the engine would have served. Because the scheduler forwards immediately rather than queueing, the effective maximum burst size through PAIR is 120 s / per-request time per node, regardless of how many requests the client sends. For the 27B example that is about 10 requests per node. Workaround: keep bursts under that ceiling, or pre-load models with a direct keep_alive request before routing through PAIR.

Sanitized logs

One line per failed request, queue-depth case:

12:04:58.xxx [ollama-proxy] DEBUG proxy request complete id=... node_id=a9489f3b-... method=POST path=/v1/chat/completions target=127.0.0.1:11434 status=502 duration_ms=120002 ttfb_ms=0 err="net/http: timeout awaiting response headers"

Cold-load case:

11:00:20  request sent
11:02:20.115 [ollama-proxy] DEBUG proxy request complete id=65 node_id=70506f7d-... method=POST path=/v1/chat/completions target=192.168.1.32:11435 status=502 duration_ms=120002 ttfb_ms=0 err="net/http: timeout awaiting response headers"

Full request logs, the load generator, and GPU samples are available on request.

主要言語
Go
スター
1.6k
フォーク
266
平均マージ
3日 15時間
マージ済み PR（30日）
23

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/Personal-AI-Router のほかの issue

NVIDIA/Personal-AI-Router の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。