How do I transform relative bounding boxes after cropping?

Question

I need to use bounding boxes in this format, according to tf.image.draw_bounding_boxes:

[y_min, x_min, y_max, x_max]

I have this image of shape (720, 1280, 3) (full size image here):

And I have bounding boxes:

array([[0.134722 , 0.425    , 0.327778 , 0.5      ],
       [0.294444 , 0.3140625, 0.444444 , 0.4124995],
       [0.473611 , 0.294531 , 0.636111 , 0.417969 ],
       [0.5972225, 0.392968 , 0.7986115, 0.497656 ],
       [0.5416665, 0.486719 , 0.6999995, 0.645313 ],
       [0.3986115, 0.425    , 0.5319445, 0.546094 ],
       [0.3499995, 0.5726565, 0.5041665, 0.6710935],
       [0.215278 , 0.507031 , 0.383334 , 0.590625 ]])

It works with the rectangle image:

image = plt.imread(myimage) 
x = tf.image.draw_bounding_boxes(image[None, ...], 
                             np.array(bboxes)[None, ...], 
                             [[255., 0., 0.] for i in range(len(bboxes))])
plt.imshow(x[0, ...].numpy().astype(np.uint8))
plt.show()

But it's all failing when I crop my picture to a square:

image = plt.imread(myimage)[:, 280:-280]
x = tf.image.draw_bounding_boxes(image[None, ...], 
                             np.array(bboxes)[None, ...], 
                             [[255., 0., 0.] for i in range(len(bboxes))])
plt.imshow(x[0, ...].numpy().astype(np.uint8))
plt.show()

How can I have the right bounding boxes after cropping to a square? I'm looking to have a 640x640 square, then I will resize to 320x320.

**Edit: ** This worked:

 h, w, c = image.shape
    h_pad = 640 - h
    w_pad = 640 - w
    image = np.pad(image, [[h_pad//2, h_pad//2], 
                           [w_pad//2, w_pad//2], [0, 0]], mode='edge')
    train_images_np.append(image) 
    
    annotations = []
    labels_temp = []
    with open(annot_path) as txtfile:
        reader = csv.reader(txtfile, delimiter=',')
        for row in reader:
            annotations.append(list([(h*(2*float(row[0])-1)/640+1)/2, 
                                     (w*(2*float(row[1])-1)/640+1)/2, 
                                     (h*(2*float(row[2])-1)/640+1)/2, 
                                     (w*(2*float(row[3])-1)/640+1)/2
                                    ]))

user_6485016 · Answer

It's because you are not considering the resize of boundary boxes.
First, You are getting boundary boxes from the original image and then you are resizing boxes based on your square image.
So by doing this all your boundary box will shrink which the same amount as I can see from the output image.
Solution: do not re-size boundary boxes use delta to replace their x coordinates

delta = (width_orginal_image - width_new_image)//2
for each_boundary_box in all_boundary_box:
    new_box_x = each_boundary_box[0] - delta

subtract this delta from x cords of boundary boxes

How do I transform relative bounding boxes after cropping?

1 Answers