Reposition camera to object & lookAt() (React three fiber)

I am trying to transition camera.position and camera.lookAt smoothly between "zoomed out" and "zoomed in" views of individual, randomly placed objects.

The positioning works great. Lerping the lookAt(), however, doesn't seem to be playing nicely with other solutions for traditional ThreeJS ( see @bovesan's answer here) nor addressed by the relevant example on the react-three-fiber docs (link).

Zooming in past the z axis flips the camera around, and at the corners it's wildly distored.

You can see my progress here : https://codesandbox.io/s/three-fiber-zoom-to-object-rlme0?file=/src/App.js

With the relevant bit of code being in App.js on line 63 :

 useFrame((state) => {
    const step = 0.05;

    // `focus` is a state variable that sends a Vec3 of the objects position
    zoom ? vec.set(focus.x, focus.y, focus.z + 0.2) : vec.set(0, 0, 5);

    // HERE, looking for a way to lerp camera lookAt in a way that can toggle.
    state.camera.lookAt(0, 0, 0);
    state.camera.position.lerp(vec, step);

    state.camera.updateProjectionMatrix();
  });

I've spent hours looking for relevant examples/tutorials, but haven't come up with much. I'm afraid I don't have enough ThreeJs experience to be looking in the right direction, though, so any help in any direction would be most welcome.