Cloud first-use: GLiNER2.5-Decide waits 21s before PROVISIONING; clarify readiness and latency expectations
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 25/100
- issue の種類
- ドキュメント
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 領域
- api, cloud, documentation, performance
調査の方向性
Start with the model page that shows the 13.1 ms figure and the Cloud retry docs at sie-relay#retry-while-capacity-starts and the /v1/models state section of the API reference. Reproduce the 503 PROVISIONING response from POST /v1/decisions using the bodies in the issue, and separate time-to-headers from time-to-full-body. Done means the docs state the measurement boundary and first-use expectations, and the 21-second pre-refusal wait is either explained or filed as a separate server-side investigation.
索引モデルが issue の本文から書いたものです。
説明
Small Cloud requests had substantially different first-use and subsequent latency. In particular, fastino/GLiNER2.5-Decide took 21.306 s before returning 503 PROVISIONING with Retry-After: 60. Later identical requests succeeded in 233–244 ms, with 7.6–10.6 ms reported inference time.
Provisioning is documented and expected. This report concerns the first-use experience: a client spends 21 seconds waiting before learning it must wait again, and a catalog entry does not make the likely wait clear. The impact is an unpredictable delay before an interactive search/classification feature can produce its first result.
Observed against https://api.superlinked.com on 2026-10-25 UTC, from one remote desktop client. Successful initial responses reported SIE version 0.10.1.
| Model / tiny input | Initial observed call | Two subsequent calls, response fully read | Subsequent X-Inference-Time |
|---|---|---|---|
Qwen/Qwen3-Embedding-4B, 2 short texts |
200, 25.192 s | 563–783 ms | 25.5–26.6 ms |
Qwen/Qwen3-Reranker-4B, 2 pairs |
200, 3.971 s | 295–372 ms | 28.6–28.8 ms |
fastino/GLiNER2.5-Decide, 1 choice question |
503, 21.306 s; retry after 60 s | 200, 233–244 ms | 7.6–10.6 ms |
zai-org/GLM-OCR, small printed image |
200, 4.000 s | 287–392 ms | 52.2–59.9 ms |
docling, 1-page PDF |
200, 11.366 s | 391–419 ms | 141.5–156.7 ms |
The text calls started around 16:54:11; OCR/PDF around 16:55:34. The first successful decision call, at 16:59:20, took 1.441 s and reported 813.5 ms inference. The two subsequent rounds ran at 17:02:39–17:02:44, one request at a time, with no automatic retries; all 12 requests across six models returned 200. The sixth model, nvidia/parakeet-tdt-0.6b-v3, processed a valid 3.446-second clip in 370–508 ms subsequently (92.1–100 ms inference). Its earlier valid call is excluded from the first-use comparison because a previous invalid fixture had already contacted that model.
These are observations, not percentiles or a controlled cold/warm benchmark. Initial text calls ran concurrently across three different models; subsequent calls were sequential. Capacity was not forcibly evicted, other service load was not controlled, and the manual gap before the successful decision call does not measure provisioning duration. Timing headers are reported separately, not added together.
The GLiNER2.5-Decide page displays 13.1 ms per message, and qualifies the test farther down as a GPU server with the client alongside. The subsequent inference times above are compatible with that scale. The missing expectation is Cloud request latency and first-use waiting, rather than evidence that steady-state GPU execution is slow.
Before these calls, /v1/models returned state: "available", loaded: false for both embedding and decision models. That is consistent with catalog availability, but the API reference recommends using state without explaining this distinction in the model-list section.
Two concrete improvements would help:
- Put the measurement boundary beside the model page's headline time: colocated benchmark versus Cloud request latency, plus model-specific first-use/capacity-start expectations where known.
- Make first-use readiness actionable: clearly distinguish catalogued/loaded/starting states and investigate whether the 21-second wait before a retryable refusal can be shortened. Clarify whether a returned 60-second hint is a default, an estimate, or the compatibility-route cap, so a caller can present a useful waiting state.
For reproduction, send these exact bodies with normal Cloud authentication, Content-Type: application/json and Accept: application/json. Authentication is intentionally omitted. Measure time to headers and to the fully read response separately. If capacity is starting, honor Retry-After before one manual follow-up; do not force eviction or repeatedly poll inference.
POST /v1/encode/Qwen%2FQwen3-Embedding-4B (220 bytes):
{"items":[{"id":"garden","text":"Plant basil on Saturday and water the seedlings every morning."},{"id":"software","text":"The development server crashes when I edit a source file."}],"params":{"output_types":["dense"]}}
POST /v1/decisions (248 bytes):
{"model":"fastino/GLiNER2.5-Decide","input":"The development server crashes when I edit a source file.","questions":[{"type":"choice","name":"topic","instructions":"What is the topic?","choices":[{"value":"gardening"},{"value":"software issue"}]}]}
Request correlation:
- Initial embedding, 16:54:11.347:
01a173fb-e9e9-7f00-907b-4a443109c2d2. - Initial decision refusal, 16:54:11.366:
PROVISIONING,Retry-After: 60, served locally; no request ID in the captured response headers. - First successful decision, 16:59:20.836:
01a17400-a2f6-74c0-87c9-0e40ed0d5c47. - Subsequent decision, 17:02:43.375:
01a17403-b912-7c62-a02e-b4541053b455.
Related: #293 concerned workers repeatedly scaling down during loading; these models subsequently served successfully. #688/#698 already improved provisioning messages and retry hints. This report asks about the remaining Cloud first-use experience, not reopening those failures.
- 主要言語
- Python
- スター
- 3.5k
- フォーク
- 328
- 平均マージ
- 16時間 25分
- マージ済み PR(30日)
- 121
環境構築
このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
superlinked/sie のほかの issue
-
MODEL_NOT_FOUND gives no suggestion for a model listed only as model:profile対応中かも @ShaiShmuel が今日担当しました。 オープンenhancement rust
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
superlinked/sie#985 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 42/100
superlinked/sie#969 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
superlinked/sie#903 · コメント 4 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
superlinked/sie#892 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 20/100
superlinked/sie#840 ·
メンテナーはふだん 1 日以内に返信
superlinked/sie の issue をすべて見る
似ている issue
-
Feature Request
難易度 1/5 1〜3時間 初心者へのやさしさ 68/100
spotDL/spotify-downloader#2839 ·
メンテナーはふだん 1 日以内に返信
-
Missing license fileオープン
難易度 1/5 1時間未満 初心者へのやさしさ 85/100
-
ops/setup.py writes ops/hub.json non-atomically (the torn-write class of #219, in the installer)オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 79/100
mattabate/life-distributable#230 ·
メンテナーはふだん 1 日以内に返信
-
enhancement triage:not-checked
難易度 2/5 1〜3時間 初心者へのやさしさ 66/100
inventree/InvenTree#13318 · リアクション 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
google/secops-wrapper#229 ·