Add video player to a GLB model with controls

Question

I'm trying to add a video player in a GLB model mesh using Three.JS. Although this can done using VideoTexture, I'm trying to add all of the controls that a video player has by default.

Here's a code snippet by using VideoTexture:

let video = document.createElement( 'video' );
video.loop = true;
video.crossOrigin = 'anonymous';
video.muted = "muted";
video.load(); 
video.src = videoD;
video.controls = true

let texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.NearestFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;
let material   = new THREE.MeshBasicMaterial( { map : texture } );

// and then use this material as the material for the glb child mesh

The problem with the above approach is I can't add controls. I'm using react currently, so another way is to use HTML component and try to load that in place of the mesh, here's the approach for that using react refs

<>

and then use this ref to modify the video player position

// I'm actually not sure what I'm doing here

react_player_ref.current.position.copy(meshChild.position)
react_player_ref.current.quaternion.copy(meshChild.quaternion)
react_player_ref.current.matrix.copy(meshChild.matrix)
react_player_ref.current.matrixWorld.copy(meshChild.matrixWorld)
react_player_ref.current.scale.copy(meshChild.scale)
react_player_ref.current.rotation.set(meshChild.rotation)

user_10653440 · Answer

I think your best bet might be to build a UI yourself out of a bunch of planes, and use raycasting or libraries such as this to register clicks.

Since your threeJS video texture uses a video dom element, it should be easily controllable using JavaScript.

const DemoScene = () => {
  const ref = useRef()

  useEffect(() => {
    // ...

    // setup UI planes
    const geometry = new THREE.PlaneGeometry(5, 20, 32)
    const material = new THREE.MeshBasicMaterial({
      color: 0xffff00,
      side: THREE.DoubleSide
    })
    const playUIPlane = new THREE.Mesh(geometry, material)
    scene.add(playUIPlane)

    // ...

    // setup raycasting, see documentation…
    const raycaster = new THREE.Raycaster()
    raycaster.setFromCamera(pointer, camera)
    const intersects = raycaster.intersectObjects(scene.children)

    // ...

    // check if/which UI has been intersected
    const video = ref.current
    const { object } = intersects[0]

    if (object === playUIPlane) video.play()
    if (object === pauseUIPlane) video.pause()
    if (object === incVolumeUIPlane) video.volume += 0.2

    // ...
  }, [ref])

  return (
    <>

Add video player to a GLB model with controls

1 Answers