Chart.js Point image on hover

I have set pointStyle to an image in a Chart.js line chart, but I'm looking to have the point hidden until it's hovered.

This is where I create the image variable:

var point = new Image();
point.src = 'path/to/image.png';

This is my options object:

{
    type: 'line',
    data: {
        labels: ['13 Oct 2019', '14 Oct 2019', '15 Oct 2019', '16 Oct 2019'],
        datasets: [{
            label: 'paid',
            borderColor: '#ffffff',
            data: [5, 35, 15, 45],
            fill: false,
            pointStyle: point
        }]
    }
}

This shows the image as the point, but of course not only on hover. I'd like to hide the point completely until it's hovered, and when hovered only show that point.

When not using an image, these are the options I used to show the point only on hover:

{
    type: 'line',
    data: {
        labels: ['13 Oct 2019', '14 Oct 2019', '15 Oct 2019', '16 Oct 2019'],
        datasets: [{
            label: 'paid',
            borderColor: '#ffffff',
            data: [5, 35, 15, 45],
            fill: false,
            pointBackgroundColor: 'transparent',
            pointBorderColor: 'transparent',
            pointHoverBorderColor: '#ffffff',
        }]
    }
}

I also thought about showing a 1 pixel transparent image as the point until it's hovered, but I'm not sure how I could change the image on hover either.