I have a problem in doing interactive visualization (in python). It seems that there is no API can both intercept key-events and mouse-events, so I turn to use open3d.visualization.gui to implement everything manually. For the mouse-event, I got the pixel coordinate from the API, but I have difficulty in turning it into the model coordinate.
I noticed the open3d.visualization.rendering.Camera.unproject method, but it didn't give me reasonable outcomes... What made me more confused was when I changed the viewpoint, the view_matrix was supposed to change as well, for example, if I pulled away the camera, the translation part of the matrix was supposed to change. However, it stayed the same all the time. Could anyone help me with either
- turning a (e.x, e.y) (pixel coordinate) to model coordinate or
- explaining how unproject method works or
- obtaining the right view_matrix of the camera.
Thanks!