I am trying to create a tensor point cloud with open3D, so I can process it on my GPU, but I can't seem to make it work. The depth image is captured from a realsense D435 camera and the depth data is stored as a numpy array as requested by the docs. I keep getting an error where it won't accept my intrinsic or depth image.
Note: It has been working perfectly with the normal o3d.geometry.PointCloud.create_from_depth_image() on my cpu, but something about the tensor input is throwing off the program.
line 107, in get_point_cloud_from_frame point_cloud = o3d.t.geometry.PointCloud.create_from_depth_image(img_depth, intrinsic_mat, depth_trunc=0.75 * depth_multiplier) TypeError: create_from_depth_image(): incompatible function arguments. The following argument types are supported: 1. (depth: open3d::t::geometry::Image, intrinsics: open3d.cpu.pybind.core.Tensor, extrinsics: open3d.cpu.pybind.core.Tensor = [[1.0 0.0 0.0 0.0], [0.0 1.0 0.0 0.0], [0.0 0.0 1.0 0.0], [0.0 0.0 0.0 1.0]] Tensor[shape={4, 4}, stride={4, 1}, Float32, CPU:0, 0x38164ed0], depth_scale: float = 1000.0, depth_max: float = 3.0, stride: int = 1, with_normals: bool = False) -> open3d.cpu.pybind.t.geometry.PointCloud
Invoked with: Image[size={1080,1920}, channels=1, UInt16, CPU:0], [[1366.19 0.0 971.087], [0.0 1364.69 545.059], [0.0 0.0 1.0]] Tensor[shape={3, 3}, stride={3, 1}, Float64, CPU:0, 0xa4941920]; kwargs: depth_trunc=4.5
My code:
depth_data = o3d.core.Tensor(np.asarray(frame.depth_data))
img_depth = o3d.t.geometry.Image(depth_data)
depth_multiplier = 6
intrinsic_mat = o3d.core.Tensor(np.array([[1366.19, 0.0, 971.087], [0.0, 1364.69, 545.059], [0.0, 0.0, 1.0]]))
print(intrinsic_mat)
point_cloud = o3d.t.geometry.PointCloud.create_from_depth_image(img_depth, intrinsic_mat, depth_trunc=0.75 * depth_multiplier)
point_cloud.cuda()
For the depth data, any 1920x1080 depth image can be used