D3-zoom - View jumps when switching between drag and pan

I'm having an issue with d3-zoom that's racking my brain.

Here's a codepen: https://codepen.io/parliament718/pen/BaNQPXx

The Y-axis has the ability to drag it to rescale.

The problem is if I drag/rescale it, and then PAN the chart afterward, there is an unwanted "jump".

I'm not understanding how to keep my 2 zoom behaviors in sync.

const zoom = d3.zoom()
  .on('zoom', () => {
    const t = d3.event.transform;
    x.domain(t.rescaleX(x2).domain());
    y.domain(t.rescaleY(y2).domain());
    render();
  });

const yAxisZoom = d3.zoom()
  .on('zoom', () => {
    y.domain(d3.event.transform.rescaleY(y2).domain());
    render();
  });

const yAxisDrag = d3.drag()
  .on('drag', () => {
    const factor = Math.pow(2, -d3.event.dy * 0.01);
    d3.select('#zoom-chart .plot-area').call(yAxisZoom.scaleBy, factor);
  });