Matrix from cv2.getPerspectiveTransform does not transform points appropriately

Viewed 75

The function cv2.getPerspectiveTransform calculates the 3x3 matrix of a perspective transform from four pairs of corresponding points.

I have an image of a pool table with perspective distortion where, upon manually selecting the four corners (in the order of top left, top right, bottom left, bottom right), cv2.getPerspectiveTransform calculates the 3x3 matrix necessary to transform the corners of the pool table to the corners of the displayed window. I then apply cv2.warpPerspective to undistort the pool table image with the calculated matrix.

I want to use this matrix to transform individual points from the distorted pool table image to the undistorted pool table image. When I attempt to transform the same manually selected corners of the distorted pool table with the transformation matrix by multiplying the 3x3 transformation matrix with the corner coordinates (written as (x, y, 1) per the linked documentation), the transformed corner points do not match the corners of the displayed window as expected (shown below). That is, the red dots should be at the corner of the image but are not.

Failed Corner Transformation

Can someone explain why the transformed corner points do not match the corners of the displayed window?

Attached below is my code:

import cv2
import numpy as np


def mouse_position(event, mouse_x, mouse_y, flags, param):
    global corners
    if event == cv2.EVENT_LBUTTONDOWN:
        print(f"Corner #{len(corners) + 1}: {(mouse_x, mouse_y)}")
        corners.append((mouse_x, mouse_y))


corners = []
img = cv2.imread("Assets/Setup.jpg")
cv2.namedWindow("Select Corners of Pool Table")
cv2.setMouseCallback("Select Corners of Pool Table", mouse_position)
while len(corners) < 4:
    cv2.imshow("Select Corners of Pool Table", img)
    if cv2.waitKey(1) != -1:
        print("Exiting program.")
        cv2.destroyAllWindows()
        exit(0)
cv2.destroyAllWindows()
corners = np.array(corners, dtype="float32")
height, width = img.shape[:2]
transform_matrix = cv2.getPerspectiveTransform(corners, np.array([(0, 0), (width, 0), (0, height), (width, height)],
                                                                 dtype="float32"))
undistorted_image = cv2.warpPerspective(img, transform_matrix, (width, height))
transformed_corners = (transform_matrix @ np.hstack((corners, np.ones((corners.shape[0], 1)))).T).T.astype(int)
for corner in transformed_corners:
    cv2.circle(undistorted_image, (corner[0], corner[1]), 30, (0, 0, 255), -1)
cv2.imshow("Frame", undistorted_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
1 Answers

You can use cv2.perspectiveTransform to transform points.

If you want to do this manually, by multiplying (x,y,1) with the 3x3 transformation matrix, make sure that you divide the result by its z value, so it will be (x',y',1) and is a homogeneous point representation.

Related