Jupyter notebook: Register mouse position clicks (moves) on a displayed image (jpg/png)

Question

Using Python 3.6+, Jupyter notebooks and matplotlib, I want to find out how to get the x,y position of an image by moving the mouse over it and/or clicking the position

I'm using any image, for example a png sized 966 x 525 pixel.

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

fig = plt.figure(figsize=(20,30))

img=mpimg.imread('ausgabe.png')
imgplot = plt.imshow(img)
plt.show();

Many suggested solutions on stackoverflow involve connecting an event to matplotlib, like

fig.canvas.mpl_connect('button_press_event', onclick)

(see Store mouse click event coordinates with matplotlib)

But in Jupyter that just doesn't react. Instead clicking on the image sometimes enlarges it.

What is a good way to display a png in Jupyter so that I could collect (=print) the click positions - or rerun the cell with the collected information?

user_5156102 · Answer

you can use the native Tk backend and you can retrieve the mouse click position this way. for example

%matplotlib tk
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

fig = plt.figure(figsize=(20,30))

img=mpimg.imread('ausgabe.png')

def onclick(event):
    ix, iy = event.xdata, event.ydata
    print(ix, iy)

cid = fig.canvas.mpl_connect('button_press_event', onclick)

imgplot = plt.imshow(img)
plt.show()

A separate tk window will pop-up and when you click, the x,y position will print in the notebook. if that doesn't work, you might need to use %matplotlib qt5 or even %matplotlib qt4

user_2557276 · Answer

I agree with the post above about switching to tk. The only thing I would like to add is that in order to get point clicks on an image, one can simply use ginput().

import matplotlib
matplotlib.use('TkAgg')

pil = Image.open("..\img\japan.jpg")
im = array(pil)
imshow(im)

pts = ginput(3) #number of clicks
print(pts)

Result: [(0.052688172043010795, 0.06725720343611918), (1.9451612903225803, 0.8951697538724439), (3.9854838709677414, -0.7310870416274797)]

user_6737387 · Answer

As pointed out by @ImportanceOfBeingErnest.

Add %matplotlib notebook to your program and you would be able to register clicks and interact with the graph like zoom in / out, move and reset. Here's how the result would appear.

Before:

After:

Jupyter notebook: Register mouse position clicks (moves) on a displayed image (jpg/png)

3 Answers