Approximating a quadrilateral from a given mask

Question

Goal:

I'd like to estimate a 4 coordinates quadrilateral (not only rectangles) of a given masked object as shown in the image + without losing any pixel of the masked object.

Trials:

I tried using CV2 however couldn't end up with a solution.

cv2.boundingRect: returns the coordinates of the bounding rectangle (while the quadrilateral estimation is not always necessary to be a perfect rectangle)
cv2.findContours + cv2.approxPolyDP: isn't that accurate and returns an estimate extreme points of the object (Needs more work to estimate the quadrilateral 4 coordinates and there might be an easier and faster solution).

Code Snippets:

Trying cv2.boundinRect:

#mask = grayed image with only a specific object being masked
#image = the original rgb image
x,y,x_width,y_height = cv2.boundingRect(mask)
image=np.array(im[0])
cv2.rectangle(image,(x,y),(x+x_width,y+y_height),(0,255,0),2)
plt.imshow(image)

Trying cv2.findContours + cv2.approxPolyDP:

#mask = grayed image with only a specific object being masked
#image = the original rgb image
contours, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
selected_contour = max(contours, key=lambda x: cv2.contourArea(x))
approx = cv2.approxPolyDP(selected_contour, 0.0035 * cv2.arcLength(selected_contour, True), True)
cv2.drawContours(image, [approx], 0, (0, 0, 255), 5)
plt.imshow(image)

user_2227070 · Answer

I am not sure if there is a better or built-in version; but i have a simple idea based on random numbers:

I only did this for the top, but you can do the same for other sides. The idea is to find the bounding-box of object first; and then divide the object into equal parts so that we can find the highest peaks.

In each range, You can find points randomly; But for best results, it is best to check all the top points of the shape to find the highest peaks correctly.

After finding the highest peaks, we have to calculate a line equation with respect to those 2 points so that we can draw a global line with respect to that line equation.

import sys
import cv2
import random
import numpy as np
from tqdm import tqdm


def rndPt(l, t, r, b):
    # Generate a random point in given ROI
    return (random.randint(int(l), int(r)), random.randint(int(t), int(b)))


def intArr(arr):
    # Cast each item of 1D array to integer
    return [int(x) for x in arr]


# Load our image
pth = sys.path[0]
org = cv2.imread(pth+'/bound.png')
im = org.copy()
H, W = im.shape[:2]

# Make mask and copy from that image
im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
bw = cv2.threshold(im, 127, 255, cv2.THRESH_BINARY)[1]
im = bw.copy()

# Find the ROI of object
cnts, _ = cv2.findContours(bw, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
cnts.sort(key=lambda x: cv2.boundingRect(x)[0])
ROI = None
for cnt in cnts:
    x, y, w, h = cv2.boundingRect(cnt)
    if w < W-1 and h < H-1:
        cv2.rectangle(bw, (x, y), (x+w, y+h), 127, 2)
        ROI = {'x': x, 'y': y, 'w': w, 'h': h, 'h2': y+h}

# We have to find the peaks; so we have to
# divide the bounding-box of shape into several
# ranges.
spaces = 5
sw = ROI['w']//spaces

# Each range can have a peak as a candidate point
candidates = [(ROI['x']+(sw*x)+sw//2, ROI['h']//2) for x in range(0, spaces)]

# Divide the object and find the highest point in
# each range
for s in tqdm(range(0, spaces)):
    l = ROI['x']+(sw*s)
    cv2.line(im, pt1=(l, ROI['y']), pt2=(l, ROI['h2']),
             color=127, thickness=2)
    for x in range(0, sw):
        for i in range(0, 200):
            pt = rndPt(l, ROI['y'], l+sw, ROI['h2']//4)
            if pt[1] < candidates[s][1] and bw[pt[1], pt[0]] == 0:
                candidates[s] = pt
l = ROI['x']+(sw*spaces)
cv2.line(im, pt1=(l, ROI['y']), pt2=(l, ROI['h2']), color=127, thickness=2)
print(candidates)

# We remove duplicate points and also sort the points
# according to the peak
candidates = list(set(candidates))
candidates.sort(key=lambda p: p[1])
print(candidates)
c = candidates

# Now that we have found two of the highest points, we can
# write a line equation for these two points
xA, xB = ROI['x'], ROI['x']+ROI['w']
x1, y1 = c[0][0], c[0][1]
x2, y2 = c[1][0], c[1][1]
m = (y2-y1)/(x2-x1)
# y=mx+b -> y-mx=b
b = y1-m*x1
yA = m*xA+b
yB = m*xB+b

# Convert images to BGR
im = cv2.cvtColor(im, cv2.COLOR_GRAY2BGR)
bw = cv2.cvtColor(bw, cv2.COLOR_GRAY2BGR)

# Make a copy of image to draw candidate points
marker = im.copy()
for p in candidates:
    cv2.circle(marker, (p[0],p[1]), 
        h//25, color=(50, 100, 200),thickness=4)

# Draw lines
cv2.line(im, pt1=intArr((xA, yA)), pt2=intArr((xB, yB)),
         color=(255, 0, 100), thickness=4, lineType=cv2.LINE_AA)
cv2.line(bw, pt1=intArr(c[0]), pt2=intArr(c[1]),
         color=(100, 0, 255), thickness=4, lineType=cv2.LINE_AA)

# Save final output
top = np.hstack((org, marker))
btm = np.hstack((bw, im))
cv2.imwrite(pth+'/out.png', np.vstack((top, btm)))

Approximating a quadrilateral from a given mask

Goal:

Trials:

Code Snippets:

1 Answers