ml5js/ml5-library

[devOps] Error: "n.videoElt.captureStream is not a function" in Safari

Open

#626 opened on Oct 14, 2019

View on GitHub
 (4 comments) (0 reactions) (1 assignee)JavaScript (905 forks)batch import
browser supportfeature requesthelp wantedyolo

Repository metrics

Stars
 (6,136 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Dear ml5 community,

I'm submitting a new issue. Please see the details below.

→ Step 1: Describe the issue 📝

Did you find a bug? Want to suggest an idea for feature? I'm receiving the following bug when trying to use the YOLO model in Safari:

Unhandled Promise Rejection: TypeError: n.videoElt.captureStream is not a function. (In 'n.videoElt.captureStream()', 'n.videoElt.captureStream' is undefined)
dispatchException — runtime.js:569

It looks like captureStream isn't supported in Safari? Is there an alternative api I can use?

Here's my code:

import * as React from 'react';
import * as ReactDOM from 'react-dom';
const ml5 = require('ml5');

interface SmartCameraState {
  isLoading: boolean;
  results: string;
  width: number;
  height: number;
}

export class SmartCamera extends React.Component<{}, SmartCameraState> {
  videoRef?: HTMLVideoElement;
  canvasRef?: HTMLCanvasElement;

  detector?: any;

  constructor(props: {}) {
    super(props);
    this.state = {
      isLoading: true,
      results: '',
      width: 640,//1280,
      height: 480//960
    };
  }

  async componentDidMount() {
    if (!this.videoRef || !this.canvasRef) { return; }

    const ctx = this.canvasRef.getContext('2d') as CanvasRenderingContext2D;
    ctx.lineWidth = 5;
    ctx.strokeStyle = "#FFFFFF";
    ctx.font = '20px Arial';
    ctx.textBaseline = 'top';

    // Create a webcam capture
    const stream = await navigator.mediaDevices.getUserMedia({ video: { width: this.state.width, height: this.state.height, facingMode: 'environment'} });
    console.log('Camera loaded');
    this.videoRef.srcObject = stream;
    await this.videoRef.play();

    const classifyVideo = () => {
      this.detector.detect(gotResult);
    }

    const gotResult = (err: any, results: {label: string, confidence: number, x: number, y: number, w: number, h: number}[]) => {
      if (this.state.isLoading) { this.setState({isLoading: false}); }

      ctx.clearRect(0, 0, this.state.width, this.state.height);
      results.forEach(result => {
        const resultStr = `${result.label} ${(result.confidence * 100).toFixed(1)}%`;
        const xpos = this.state.width * result.x;
        const ypos = this.state.height * result.y;
        const boxWidth = this.state.width * result.w;
        const boxHeight = this.state.height * result.h;
        const textWidth = ctx.measureText(resultStr).width;
        // console.log(`x: ${xpos} y: ${ypos} w: ${boxWidth} h: ${boxHeight}`);

        ctx.beginPath();
        ctx.rect(xpos, ypos, boxWidth, boxHeight);
        ctx.stroke();
        ctx.fillStyle = "#FFFFFF";
        ctx.fillRect(xpos, ypos, textWidth, 22);
        ctx.fillStyle = "#000000";
        ctx.fillText(resultStr, xpos, ypos);
      });

      // this.setState({results: JSON.stringify(results, null, 2)});
      classifyVideo();  
    }

    this.videoRef.
    this.detector = await ml5.YOLO(this.videoRef, () => classifyVideo());
    // classifyVideo();
  }
  
  render() {
    return (
      <>
        {this.state.isLoading ? <div>loading...</div> : null}
        <div>
          <video id="video" autoPlay muted loop playsInline ref={this.setVideoInputRef} width={this.state.width} height={this.state.height} style={{position: 'fixed'}}/>
          <canvas width={this.state.width} height={this.state.height} ref={ ref => ref && (this.canvasRef = ref)} style={{position: 'fixed'}}/>
        </div>
      </>
    );
  }

  setVideoInputRef = (ref: HTMLVideoElement) => {
    ref && (this.videoRef = ref) 
  }
}

Contributor guide