[safari] Voicing/SpeechSynthesis is buggy in multiple iframes

Aperta
#92 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
typescript
Ambito
frontend

Direzione di ricerca

Reproduce the supplied multi-iframe SpeechSynthesis cases in Safari and compare them with Chrome using the provided iframe examples. No repository files or tests are named; trace the utterance-queue behavior involved in the reproduction and document the conditions under which speech stops. Done means the failure mode is established and a fix or clear limitation is verified.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

dev:voicing

@Nancy-Salpepi reported this bug originally in https://github.com/phetsims/phet-io-wrappers/issues/459. We are finding that our voicing in our sims behaves quite badly when running multiple sims in iframes on the same page in safari.

Test 1:



<iframe src="https://phet-dev.colorado.edu/html/ratio-and-proportion/1.2.0-dev.31/phet/ratio-and-proportion_all_phet.html?voicingInitiallyEnabled&preferencesStorage"></iframe>
<iframe src="https://phet-dev.colorado.edu/html/ratio-and-proportion/1.2.0-dev.31/phet/ratio-and-proportion_all_phet.html?voicingInitiallyEnabled&preferencesStorage"></iframe>
<iframe src="https://phet-dev.colorado.edu/html/ratio-and-proportion/1.2.0-dev.31/phet/ratio-and-proportion_all_phet.html?voicingInitiallyEnabled&preferencesStorage"></iframe>

This consistently showed that the last iframe's voicing worked perfectly, but the other two were silent. Behavior is great on chrome, with all three iframes working.

Then I added @marlitas and @samreid (our resident safari devices holders) and we made this test:

iframe contents:


<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8"/>
  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/>
  <meta name="viewport" content="width=device-width"/>

  <title>Speech synthesiser</title>

  <link rel="stylesheet" href="style.css"/>
</head>

<body>
<h1>Speech synthesiser</h1>

<p>
  Enter some text in the input below and press return or the "play" button
  to hear it. change voices using the dropdown menu.
</p>

<form>
  <label for="txt">Enter text</label>
  <input id="txt" type="text" class="txt"/>
  <div>
    <label for="rate">Rate</label
    ><input type="range" min="0.5" max="2" value="1" step="0.1" id="rate"/>
    <div class="rate-value">1</div>
    <div class="clearfix"></div>
  </div>
  <div>
    <label for="pitch">Pitch</label
    ><input type="range" min="0" max="2" value="1" step="0.1" id="pitch"/>
    <div class="pitch-value">1</div>
    <div class="clearfix"></div>
  </div>
  <select></select>
  <div class="controls">
    <button id="play" type="submit">Play</button>
  </div>
</form>

<script>
  const synth = window.speechSynthesis;

  const inputForm = document.querySelector( "form" );
  const inputTxt = document.querySelector( ".txt" );
  const voiceSelect = document.querySelector( "select" );

  const pitch = document.querySelector( "#pitch" );
  const pitchValue = document.querySelector( ".pitch-value" );
  const rate = document.querySelector( "#rate" );
  const rateValue = document.querySelector( ".rate-value" );

  let voices = [];

  function populateVoiceList() {
    voices = synth.getVoices().sort( function( a, b ) {
      const aname = a.name.toUpperCase();
      const bname = b.name.toUpperCase();

      if ( aname < bname ) {
        return -1;
      }
      else if ( aname == bname ) {
        return 0;
      }
      else {
        return +1;
      }
    } );
    const selectedIndex =
      voiceSelect.selectedIndex < 0 ? 0 : voiceSelect.selectedIndex;
    voiceSelect.innerHTML = "";

    for ( let i = 0; i < voices.length; i++ ) {
      const option = document.createElement( "option" );
      option.textContent = `${voices[ i ].name} (${voices[ i ].lang})`;

      if ( voices[ i ].default ) {
        option.textContent += " -- DEFAULT";
      }

      option.setAttribute( "data-lang", voices[ i ].lang );
      option.setAttribute( "data-name", voices[ i ].name );
      voiceSelect.appendChild( option );
    }
    voiceSelect.selectedIndex = selectedIndex;
  }

  populateVoiceList();

  if ( speechSynthesis.onvoiceschanged !== undefined ) {
    speechSynthesis.onvoiceschanged = populateVoiceList;
  }

  function speak() {
    // if ( synth.speaking ) {
    //   console.error( "speechSynthesis.speaking" );
    //   return;
    // }

    if ( inputTxt.value !== "" ) {
      const utterThis = new SpeechSynthesisUtterance( inputTxt.value );

      utterThis.onend = function( event ) {
        console.log( "SpeechSynthesisUtterance.onend" );
      };

      utterThis.onerror = function( event ) {
        console.error( "SpeechSynthesisUtterance.onerror" );
      };

      const selectedOption =
        voiceSelect.selectedOptions[ 0 ].getAttribute( "data-name" );

      for ( let i = 0; i < voices.length; i++ ) {
        if ( voices[ i ].name === selectedOption ) {
          utterThis.voice = voices[ i ];
          break;
        }
      }
      utterThis.pitch = pitch.value;
      utterThis.rate = rate.value;
      console.log( inputTxt.value );
      synth.speak( utterThis );
    }
  }

  inputForm.onsubmit = function( event ) {
    event.preventDefault();

    let count = 0;
    const id = setInterval( () => {
      speak();
      count++;

      if ( count >= 5 ) {
        clearInterval( id );
      }
    }, 1000 );

    inputTxt.blur();
  };

  pitch.onchange = function() {
    pitchValue.textContent = pitch.value;
  };

  rate.onchange = function() {
    rateValue.textContent = rate.value;
  };

  voiceSelect.onchange = function() {
    speak();
  };
</script>
</body>
</html>

outer webpage:

<html>
<body>
<iframe src="html.html" width="800" height="400"></iframe>
<iframe src="html.html" width="800" height="400"></iframe>
</body>
</html>

In that we got semi sporadic behavior where it is possible that when only interacting with iframe 1, we hear a single voiced alert and then never hear anything again.

It is also possible that we hear everything from iframe 1, but then when go to use the second iframe, we hear just a single voiced alert, and then never anything again. We will need to investigate more, but I believe this should block voicing sims until we learn more about it.

Lingua principale
TypeScript
Stelle
1
Fork
5
Nessuna PR unita negli ultimi 30g

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di phetsims/utterance-queue

Tutte le issue di phetsims/utterance-queue

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.