What's the most efficient way to acquire the webcam stream directly into WebGL as a texture?
The standard process (as documented in Three.js) is to use the video tag, then manipulate it in a canvas tag and then from canvas to WebGL.
This increases CPU overhead and Chrome spins up a worker to send image data from the camera to the GPU (e.g. 30 times per second).
Is there any way to do this more efficiently and reduce rendering/processing on the CPU?
The easiest way would be to simply put the stream in the video HTML element and then transform it into the texture. This way results in low fps. Luckily there is a library that does it in a quick way. I don't understand how exactly though (probably extract the texture from the stream itself, or optimize the first method).
https://p5js.org/examples/dom-video-capture.html https://p5js.org/examples/3d-shader-using-webcam.html#
You can check source code to understand how they do it. https://github.com/processing/p5.js/
When I compared these two approaches, the second one resulted in 60fps, while the first one was 20fps for my shader.
