Projection of a 3D point into 2D plan on Python : problems with Intrinsic/Extrinsic matrix in Open3D

Viewed 19

I'm using Open3d on Python and my goal was the following : place a sphere in the 3D space, take a screenshot with the Open3D camera and then creating a bounding box around the sphere with CV2. Here are the grey sphere coordinates :

sphere_coordinate = [0.5, 0.5, 0.5]

Then I use the intrinsic and extrinsic parameters, to create the matrices and compute the coordinates : `{

"extrinsic" : 
[
    0.9798753907884673,
    -0.023315492319296981,
    0.19824380531319841,
    0.0,
    0.021669035311339107,
    0.99971036496253884,
    0.010470868881974344,
    0.0,
    -0.19843052042421405,
    -0.0059643947200334804,
    0.98009680876931715,
    0.0,
    -1.9176285699106663,
    -1.9110189098067627,
    0.095854438451187352,
    1.0
],
"intrinsic" : 
{
    "height" : 1080,
    "intrinsic_matrix" : 
    [
        935.30743608719376,
        0.0,
        0.0,
        0.0,
        935.30743608719376,
        0.0,
        959.5,
        539.5,
        1.0
    ],
    "width" : 1920
},

}`

Then I run my code but I found pixel either outside of the image or not in the right position at all. With this example I've found :

array([1935, 331)

Here is my code :

with open(os.path.abspath('open3d_data/room_test.json')) as json_file:
   data_camera = json.load(json_file)
extrinsic_matrix = np.array([[element for element in data_camera["extrinsic"][i:i+4]] for i in 
                    range(0,16,4)])
extrinsic_matrix = np.transpose(extrinsic_matrix)
intrinsic_matrix = np.array([[element for element in data_camera["intrinsic"] 
                    ["intrinsic_matrix"][i:i+3]] for i in range(0,9,3)])
intrinsic_matrix = np.concatenate((np.transpose(intrinsic_matrix), np.array([[0], [0], [0]])), 
                   axis=1)
full_transfo_matrix = np.matmul(intrinsic_matrix, extrinsic_matrix)
array_3d_augmented = np.array(sphere_coordinate + [1])
array_2d_pixel = np.matmul(full_transfo_matrix, array_3d_augmented)

Basically I've done that but this is not working as expected in Open3D. This for;ula is supposed to be true. Any guess of what's going on ? The formula I've followed

Thanks a lot.

0 Answers
Related