I have been searching for days after a possible (alternative) solution to determine a camera's distortion coefficients WITHOUT the use chessboards/checkerboard. The reason for this is that I have sufficiently many points for which I know both the real-world position and the 2D-image position and I image it could be done using this information.
So, here is what I know about:
Camera
focal_length = 3.9
image_height = 3456
image_width = 4608
esl_height = 35
sensor_height = 4.69
sensor_width = 6.26
pixel_size = 1.34/1000
FOV_x = 71.9
FOV_y = 56.7
F_x = image_width /(2 * tan(FOV_x* pi / 360))
F_y = image_height /(2 * tan(FOV_y* pi / 360))
C_x = image_width / 2
C_y = image_height / 2
row1 = [F_x,0,C_x]
row2 = [0,F_y, C_y]
row3 = [0,0,1]
camera_matrix = np.matrix([row1,row2,row3])
which gives:
matrix([[3.17701054e+03, 0.00000000e+00, 2.30400000e+03],
[0.00000000e+00, 3.20254589e+03, 1.72800000e+03],
[0.00000000e+00, 0.00000000e+00, 1.00000000e+00]])
Data
I have a dataset containing 49 points for which I have both the real-world (x,y) positions (I also have the z values, but I am really only interested in the (x,y) position. Also, I have the pixel positions for all these points in an image (They are the bounding box centers in the object deterction I perform.
id manual.location.x manual.location.y manual.location.z \
0 0 32.9878 41.9033 0.215
1 1 32.9878 44.4404 0.215
2 2 32.9878 44.4404 0.215
3 3 32.9878 44.4404 0.565
4 4 32.9878 44.4404 0.565
5 5 32.9878 44.4404 0.565
6 6 32.9878 44.4404 0.565
7 7 32.9878 44.4404 0.565
8 8 32.9878 41.9033 0.753
9 9 32.9878 44.4404 0.779
10 10 32.9878 44.4404 0.779
11 11 32.9878 44.4404 0.779
12 12 32.9878 41.9033 1.024
13 13 32.9878 44.4404 1.024
14 14 32.9878 42.7490 1.100
15 15 32.9878 43.5947 1.100
16 16 32.9878 43.5947 1.100
17 17 32.9878 43.5947 1.100
18 18 32.9878 43.5947 1.100
19 19 32.9878 44.4404 1.231
20 20 32.9878 44.4404 1.231
21 21 32.9878 44.4404 1.231
22 22 32.9878 44.4404 1.231
23 23 32.9878 42.7490 1.320
24 24 32.9878 43.5947 1.412
25 25 32.9878 42.7490 1.412
26 26 32.9878 43.5947 1.412
27 27 32.9878 44.4404 1.412
28 28 32.9878 44.4404 1.458
29 29 32.9878 44.4404 1.458
30 30 32.9878 44.4404 1.458
31 31 32.9878 44.4404 1.458
32 32 32.9878 42.7490 1.620
33 33 32.9878 42.7490 1.620
34 34 32.9878 43.5947 1.620
35 35 32.9878 43.5947 1.620
36 36 32.9878 42.7490 1.620
37 37 32.9878 44.4404 1.651
38 38 32.9878 44.4404 1.651
39 39 32.9878 44.4404 1.651
40 40 32.9878 44.4404 1.651
41 41 32.9878 44.4404 1.651
42 42 32.9878 42.7490 1.850
43 43 32.9878 42.7490 NaN
44 44 32.9878 44.4404 NaN
45 45 32.9878 44.4404 NaN
46 46 32.9878 44.4404 1.850
47 47 32.9878 43.5947 1.850
48 48 32.9878 44.4404 1.850
bbox_center_x bbox_center_y
0 4269.5 2914.0
1 1035.0 2883.5
2 843.5 2880.0
3 1516.5 2529.0
4 1247.5 2527.0
5 1730.0 2525.5
6 987.0 2522.0
7 765.5 2520.0
8 4435.0 2356.5
9 1196.0 2257.0
10 938.5 2255.0
11 704.5 2250.0
12 4547.0 1998.0
13 1291.0 1980.0
14 3681.0 1864.5
15 2394.0 1856.0
16 1931.5 1854.0
17 2043.0 1853.5
18 2043.0 1853.5
19 1620.0 1676.0
20 1267.0 1675.5
21 847.0 1672.5
22 593.5 1669.5
23 3783.5 1452.0
24 1931.5 1446.5
25 3346.5 1443.5
26 2878.0 1443.0
27 1632.0 1350.0
28 1409.0 1346.5
29 945.0 1339.5
30 727.0 1334.0
31 534.5 1332.0
32 4349.0 1109.0
33 3859.0 1101.0
34 1920.5 1097.5
35 2878.0 1090.5
36 3364.0 1087.0
37 1460.5 1037.0
38 1290.5 1036.0
39 1065.5 1034.0
40 748.0 1032.5
41 473.0 1029.0
42 4262.0 726.0
43 3683.5 713.0
44 1375.0 710.0
45 1094.5 708.0
46 850.0 705.0
47 3103.5 702.5
48 375.5 701.5
Pixel to real world without distortion taken into account
I have managed to get the real world positions from the pixels, but as you can imagine, the farther away I am from the image center (C_x, C_y) the farther the result of the transformation is. I assume it is due to distortion. So, what I do is this:
I define four points in a quadrilater in my image and their real-world counterparts
import numpy as np
import cv2
import scipy
coords = {
"realworld": np.array([
[32.9878,41.9033],
[32.9878,44.4404],
[32.9878,42.7490],
[32.9878,44.4404]
]),
"pixel": np.array([
[4269.5,2914.0],
[843.5,2880.0],
[4262.0,726.0],
[375.5,701.5] #
])
}
realworld_array = coords['realworld']
pixel_array= coords['pixel']
and find the Perspective Transform using opencv:
M = cv2.getPerspectiveTransform(np.float32(pixel_array),np.float32(realworld_array))
I define my function from pixels to real-world points
def R2_to_R3(pixel):
if type(pixel) != np.ndarray:
pixel = np.array(pixel).reshape(1,2)
assert pixel.shape[1]==2, "Need (N,2) input array"
pixel = np.concatenate([pixel, np.ones((pixel.shape[0],1))], axis=1)
lonlat = np.dot(M,pixel.T)
return (lonlat[:2,:]/lonlat[2,:]).T
Now, for the center of the image:
pixel = (C_x,C_y)=(2304.0, 1728.0)
R2_to_R3((C_x,C_y))
array([[32.98780048, 43.50150617]])
Real world coordinates [32.98780048, 43.5314615]
But for a point further away
pixel = (4435.0,2356.5)
R2_to_R3(pixel)
array([[32.98780025, 42.11436824]])
Real world coordinates: [32.98780025, 41.9033]
which is rather bad.
Now, note that I never use my camera matrix and this is due to the fact that I do not know how to compute the distortion coefficient. Is there a way to do this without the painful process of doing the chessboard exercice?
Grateful for anything that would improve my work.
Update
After some searching (i.e. googling), I found that this could be achieved using cv.calibrateCamera() in the following way:
image_height = 3456
image_width = 4608
w = image_width
h = image_height
rvec = (0, 0, 0) # rotation relative to the frame: Static
tvec = (0, 0, 0) # translation relative to the frame: Static
size = (w,h)
cv.calibrateCamera(
[pts3d], [pts2d],
size,camera_matrix,np.float32([0,0,0,0,0]),
flags=0
| cv.CALIB_FIX_K1
| cv.CALIB_FIX_K2
| cv.CALIB_FIX_K3
| cv.CALIB_FIX_K4
| cv.CALIB_FIX_K5
,rvec,tvec)
However, this returns:
File "<ipython-input-319-af5936255ea4>", line 31
,rvec,tvec)
^
SyntaxError: positional argument follows keyword argument
What does this mean?