I am using a checkerboard target to extrinsically calibrate an omnidirectional camera with another sensor. For that purpose, I utilize the function cv2.omnidir.projectPoints twice, and it unexpectedly returns different results.
Here I show a projection of the outer corners of the checkerboard to the image. In the first case the projection is done from the board coordinate system (CS) to the image CS. In the second case, I manually project the corners from board CS to the camera CS, and then project it to the image, using rvec=tvec=(vector of zeros).
import numpy as np
import cv2
# --- Camera parameters
K = np.array([[939.265, 0. , 965.693],
[ 0. , 942.402, 645.578],
[ 0. , 0. , 1. ]])
D = np.array([-0.156, -0.03 , 0. , 0.001], dtype=float32)
# --- Corners in board coordinate system
outer_corners_bcs = np.array([[-0.175, -0.17 , 0. ],
[ 0.8 , -0.17 , 0. ],
[-0.175, 0.67 , 0. ],
[ 0.8 , 0.67 , 0. ]])
# --- Rotation & Translation vectors from checkerboard target to camera
rvec = np.array([[-1.138, -1.421, 2.827]])
tvec = np.array([[-6.852, -5.473, 4.549]])
# --- Project from board to image
uv, _ = cv2.omnidir.projectPoints(outer_corners_bcs.reshape(1, -1, 3).astype('float64'),
rvec.reshape(1, 3),
tvec.reshape(1, 3), K, 1, D)
print(uv)
[[[556.417 320.275]
[504.347 320.397]
[562.968 272.863]
[509.031 272.395]]]
# --- Convert to camera coordinate system and project to image
R = eulerAnglesToRotationMatrix(rvec)
outer_corners_ccs = (R @ outer_corners_bcs.T + tvec).T
uv, _ = cv2.omnidir.projectPoints(outer_corners_ccs.reshape(1, -1, 3),
np.zeros(3).reshape(1, 3),
np.zeros(3).reshape(1, 3), K, 1, D)
print(uv)
[[[699.384 304.698]
[639.438 180.463]
[876.787 527.013]
[752.581 334.091]]]
My expectation is that the results will be identical.
Thanks for any help!