/v1/audio/transcriptions rejects the model name the server was started with (404 unknown_model) since 0.3.0

Open
  • Is something wrong?
#94 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
cpp
Domain
api

Research direction

Start with the API behavior described in docs/api.md, then reproduce the issue using the supplied nemo-speech serve and curl commands. Trace how /v1/models builds the ASR model ID and how /v1/audio/transcriptions validates the model field. Done means clients can use a stable, discoverable model name for transcription without receiving unknown_model.

Written by the indexing model from the issue text.

Description

Summary

Since 0.3.0, POST /v1/audio/transcriptions validates the model field against the IDs in GET /v1/models and returns 404 unknown_model for anything else (as described in docs/api.md).

However, the ID reported for the ASR model is not a name a client would know. When started with --asr-model nemotron-3.5, the server lists the model ID as .nemotron-3.5-asr-streaming-0.6b.q8_0.gguf, prepending a leading dot that the file on disk does not have. As a result, a client that sends the name used to start the server (the indexed name nemotron-3.5 or the GGUF file name) gets a 404. On 0.2.0, the same request succeeded because the model field was ignored.

/v1/audio/speech behaves differently: it accepts magpie, magpietts, and even nonsense.

Repro

nemo-speech pull nemotron-3.5
nemo-speech serve --port 18999 --no-ui --asr-model nemotron-3.5 &
curl -s http://127.0.0.1:18999/v1/models | jq '.data[].id'
# ".nemotron-3.5-asr-streaming-0.6b.q8_0.gguf"

for m in nemotron-3.5 nvidia/nemotron-3.5-asr-streaming-0.6b \
         nemotron-3.5-asr-streaming-0.6b.q8_0.gguf .nemotron-3.5-asr-streaming-0.6b.q8_0.gguf ""; do
  printf '%-45s ' "model='$m'"
  curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:18999/v1/audio/transcriptions \
    -F [email protected] -F "model=$m"
done

On 0.3.0:

model='nemotron-3.5'                          404
model='nvidia/nemotron-3.5-asr-streaming-0.6b' 404
model='nemotron-3.5-asr-streaming-0.6b.q8_0.gguf' 404
model='.nemotron-3.5-asr-streaming-0.6b.q8_0.gguf' 200
model=''                                      200

The 404 response body is {"error":{"code":"unknown_model","message":"requested model is not available","param":"model","type":"invalid_request_error"}}. On 0.2.0, all five requests return 200.

Starting the server with the full GGUF path instead of the indexed name produces the same ID, including the leading dot. The server log also prints [asr] model=.nemotron-3.5-asr-streaming-0.6b.q8_0.gguf.

Expected

Either of these would work:

  • Accept the name the model was loaded with (the indexed name such as nemotron-3.5, or the GGUF file name) in addition to the listed ID.
  • List a stable ID without the leading dot (for example, the indexed name) so clients can discover and send it.

The leading dot appears unintended in any case.

Impact

Upgrading from 0.2.0 to 0.3.0 breaks OpenAI-compatible clients that send model (which the OpenAI SDK requires) unless they first look up the leading-dot ID. We worked around this by omitting model, which the docs state selects the loaded model.

Environment

  • nemo-speech 0.3.0, macOS 26 on Apple M5, Metal build (also reproduced the ID listing on 0.2.0)
  • ASR: nemotron-3.5 (nvidia/nemotron-3.5-asr-streaming-0.6b, q8_0)
Dominant language
C++
Stars
207
Forks
47
Avg merge
13h 10m
Merged PRs (30d)
12

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/NeMo-Speech.cpp

All issues in NVIDIA/NeMo-Speech.cpp

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.