solvePnP object_points / image_points shape?

user_896112 0
Viewed 1503

I'm trying to call solvePnP with the following arguments.

apriltag_object_points = np.array([(-1, -1, 0), (-1, 1, 0), (1, 1, 0), (1, -1, 0)], dtype=np.float)

camera_matrix_left = np.eye(3)

dist_left = np.zeros((5, 1))

image_points = np.array(detection.position, dtype=np.float)

cv2.solvePnPRansac(apriltag_object_points, image_points, camera_matrix_left, dist_left)

However, I'm getting the following error:

OpenCV Error: Assertion failed (CV_IS_MAT(_src) && CV_IS_MAT(_dst) && (_src->rows == 1 || _src->cols == 1) && (_dst->rows == 1 || _dst->cols == 1) && _src->cols + _src->rows - 1 == _dst->rows + _dst->cols - 1 && (CV_MAT_TYPE(_src->type) == CV_32FC2 || CV_MAT_TYPE(_src->type) == CV_64FC2) && (CV_MAT_TYPE(_dst->type) == CV_32FC2 || CV_MAT_TYPE(_dst->type) == CV_64FC2)) in cvUndistortPoints, file /Users/travis/miniconda3/conda-bld/opencv_1506476120161/work/opencv-3.3.0/modules/imgproc/src/undistort.cpp, line 312
Traceback (most recent call last):
  File "/Users/me/Documents/Code/project/file.py", line 139, in <module>
    ret = cv2.solvePnPRansac(apriltag_object_points, image_points, camera_matrix_left, dist_left)
cv2.error: /Users/travis/miniconda3/conda-bld/opencv_1506476120161/work/opencv-3.3.0/modules/imgproc/src/undistort.cpp:312: error: (-215) CV_IS_MAT(_src) && CV_IS_MAT(_dst) && (_src->rows == 1 || _src->cols == 1) && (_dst->rows == 1 || _dst->cols == 1) && _src->cols + _src->rows - 1 == _dst->rows + _dst->cols - 1 && (CV_MAT_TYPE(_src->type) == CV_32FC2 || CV_MAT_TYPE(_src->type) == CV_64FC2) && (CV_MAT_TYPE(_dst->type) == CV_32FC2 || CV_MAT_TYPE(_dst->type) == CV_64FC2) in function cvUndistortPoints

It looks like there's a problem with the shape of my arguments, but they seem to be fine...

apriltag_object_points.shape == (4, 3)
image_points.shape == (4, 2)

Do the image points have to be homogenous? Do I just have to hstack a column vector of 1's to image_points?

opencv numpy python
1 Answers

The documentation is simply wrong for this function as it's implemented in Python. The docs for solvePnP() and solvePnPRansac() both state:

Parameters:

objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. vector<Point3f> can be also passed here.

imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points. vector<Point2f> can be also passed here.

So you assumed you were good using a (N, 3) and (N, 2) array for your inputs. However, the error code says otherwise. Let's break it down:

CV_IS_MAT(_src) && CV_IS_MAT(_dst) && (_src->rows == 1 || _src->cols == 1) && (_dst->rows == 1 || _dst->cols == 1) && _src->cols + _src->rows - 1 == _dst->rows + _dst->cols - 1 && (CV_MAT_TYPE(_src->type) == CV_32FC2 || CV_MAT_TYPE(_src->type) == CV_64FC2) && (CV_MAT_TYPE(_dst->type) == CV_32FC2 || CV_MAT_TYPE(_dst->type) == CV_64FC2)

Ok, they're numpy arrays, so we can probably discard the first two. But then in the next checks we see something interesting:

(_src->rows == 1 || _src->cols == 1) && (_dst->rows == 1 || _dst->cols == 1)

It's checking that there are only one row or one column in src and dst. In other words, it's expecting your points strictly in the second version that it states in the docs; it wants a multi-channel point array. This means that the first coordinate is on one channel of a matrix, second coordinate in the next channel, etc.

So if we simply reshape your points into a multi-channel array:

>>> apriltag_object_points = apriltag_object_points.reshape(4,1,3)
>>> image_points = image_points.reshape(4,1,2)
>>> it_works, rvec, tvec, inliers = cv2.solvePnPRansac(apriltag_object_points, image_points, camera_matrix_left, dist_left)
>>> it_works
True

It works!

Future pro-tip: in every function that OpenCV does allow points in a (npoints, ncoords) format, it also accepts them in a (npoints, 1, ncoords) format AFAIK. However, some functions only work with the latter format. Therefore, if you're using point coordinates in OpenCV with Python, it's probably best to assume your points should be in a multi-channel array, where the number of channels is the number of coordinate axes.

user_50874360
Related