Whisper Model Has Inaccessible Logits/Log Probs

Aperta
#1,855 1 commento 2 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
cpp, python

Direzione di ricerca

Riprodurre il percorso Whisper tramite la chiamata Python model.generate con return_logits_vocab=True e return_scores=True, quindi esaminare i logits di WhisperGenerationResult esposti da questa API. Il lavoro è completato quando result[0].logits contiene valori accessibili per ogni token generato, mentre il comportamento esistente di scores rimane invariato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Description:
Hello, I've been tinkering with CTranslate2 with whisper converted to ct2. Using model.generate with return_logits_vocab=True and return_scores=True I'm unable to access or print out the logits entirely. For example:

result = model.generate(..., return_logits_vocab=True, return_scores=True)
print(result[0].logits[0][0])
# Output: [cpu:0 float32 storage viewed as ]
print(result[0].logits[0][0].__array_interface__)
# Output: {'shape': (), 'typestr': '<f4', 'data': (0, False), 'version': 3}

This issue is reproducible with any input, even though the sequences are transcribed correctly.

Expected Behavior:
I would expect result[0].logits to contain accessible tensors with the logit values or log probs for each token generated.

Actual Behavior:
The logits appear to be empty or not fully accessible, as indicated by the output above.

This is super strange to me because result[0].scores[0] is completely accessible and prints out exactly as expected and I'd have to imagine that the scores attribute is just some mean or derivative of all of the log probs/logits for each token generated by CTranslate2. Which leads me to believe that this is some sort of bug rather than intentional behavior to hide the log probs.

Additional Details:

  • CTranslate2 Version: 4.5.0

  • I also tried the latest commit (6a3dc63db5608567bf554ca5dbbfdac1f7a2321f) which seemed related to the logits issues. While the commit added the logits property on print WhisperGenerationResult with print(result[0]), even there the issue persists with the same empty logits and everything else working:

    logits=[[
    [cpu:0 float32 storage viewed as ]], [
    [cpu:0 float32 storage viewed as ]], [
    [cpu:0 float32 storage viewed as ]], [
    [cpu:0 float32 storage viewed as ]], [
    [cpu:0 float32 storage viewed as ]]
    

Environment:
Python 3.11.11
MacOS Sequoia 15.2

Running cpu float32 inference on an M1 Pro Macbook.

Please let me know if you need any more information. If this is indeed a bug (and not intended behavior for the Whisper model), I'm happy to investigate further or contribute a fix.

Thanks!

Lingua principale
C++
Stelle
4.8k
Fork
557
Merge medio
1g 6h
PR unite (30g)
24

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di OpenNMT/CTranslate2

Tutte le issue di OpenNMT/CTranslate2

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.