HTML/Javascript Webcam Video and Picture with different resolution

Question

I am developing an app in HTML and typescript/javascript (Chrome support only as it is embedded in an electron app at the end) where there is a Webcam (Logitech B910) video streamed with a quite low video resolution (640 x 480) in order to be able to do some post processing (mainly Tensorflow & Mediapipe) without consuming to much resources on the computer.

From time to time, I also need to some image captures with the highest Webcam resolution (1280 x 720 in my case) of the Webcam.

What is the fastest way to take a capture with a higher resolution than the curently video streamed resolution?

The code below is working but the process (mainly the double resolution switch) is very slow. (roughly 6 seconds on my computer)

Here the code detail :

1/ HTML code :

2/ Typescript Webcam video setup code :

static async startCamera(deviceId,containerSize,container : HTMLElement,targetFPS : number,_video,_canvas) {
     
     const videoConfig = {
       'audio': false,
       'video': {
           width: { min: 320, ideal: 640 , max:1280},
           height: { min: 240, ideal: 480, max:720},
           frameRate: {ideal: targetFPS}},
       'deviceId' : { exact: deviceId }
       }
     };

     const stream = await navigator.mediaDevices.getUserMedia(videoConfig);
     const camera = new Camera(_video,_canvas);
     camera.video.srcObject = stream;
     await new Promise((resolve) => {
         camera.video.onloadedmetadata = () => { resolve(_video); };
     });

     camera.video.play();

     /// Video tag size
     camera.video.width = containerSize.width;
     camera.video.height = containerSize.height;
     /// Canvas tag size
     camera.canvas.width = containerSize.width
     camera.canvas.height = containerSize.height
     //// Container tag size
     container.style.width = containerSize.width.toString()+"px"
     container.style.height = containerSize.height.toString()+"px"
        
     return camera;
   }

3/ The code I use to do the image capture :

 async takePhoto(required_width, required_height) {
    
    const track = this.video.srcObject.getVideoTracks()[0];
    let constraints :MediaTrackConstraints = track.getConstraints();
    
    /// Save the current values to load it back at the end
    const currentWidth = constraints.width;
    const currentHeight = constraints.height;
    
    /// Apply the new resolution
    constraints.width = 1920;
    constraints.height = 780;
    await track.applyConstraints(constraints);
  
    /// Do the image capture
    const capture = new ImageCapture(track);
    const { imageWidth, imageHeight } = await capture.getPhotoCapabilities();
    const width = this.setInRange(required_width, imageWidth);
    const height = this.setInRange(required_height, imageHeight);
    const photoSettings = (width && height) ? {
      imageWidth: width,
      imageHeight: height
    } : null;
    const pic = await capture.takePhoto(photoSettings);
    
    /// load back the previously current resolution saved
    constraints.width = currentWidth;
    constraints.height = currentHeight;
    await track.applyConstraints(constraints);

    return pic;
  }

setInRange(value, range) {
    if(!range) return NaN;
    let x = Math.min(range.max, Math.max(range.min, value));
    x = Math.round(x / range.step) * range.step;
    return x;
  }

Note : I thought that takePhoto() from Image Capture API is supposed to always use the highest webcam resolution but in my case (and as given by getPhotoCapabilities() result too) it always uses the resolution I applied to setup the camera (in my case 640 x 480) and that's why I need to do this 'dirty'(?) and slow process.

Is there any fastest way to do it ?

HTML/Javascript Webcam Video and Picture with different resolution

0 Answers