How to remove external hollow contours from the image without affecting the internal contours

user_7657721 0
Viewed 1326

This is a continuation of a problem under a different approach.

I want to get rid of the boxes, and everything outside of its lines, and keep whatever's inside the box, as is. Or at least only the box.

The box:

img

How I want it to look like after doing some magic:

img

This is where it was cropped from:

img

If you view the first image on a black background, you'll notice there's still a white band beyond the boxed field. It can be cropped smaller or larger by getting its bounding box's stats, and adding the padding into it.

This is the naive and my first approach. The problem here is:

  • It cannot account for different line widths too much without affecting the characters inside. Say, you put in -9 as padding. It will work mostly well enough on boxes with ~9px line width, but anything beyond the ~9px are left inside. This causes some leftover pixels, which ultimately affects the accuracy of my application.
  • On the other hand, any boxed field with line width significantly lower than 9 may then have the characters inside destroyed too.

My second attempt is removing the contour with a mask. However, it didn't turn out as expected.

The code in extracting the contours is really long so just assume stats contains the contours returned from contours, _ = cv2.findContours

# loop in each contour in stats
for i in range(len(stats)):
    # get the stats of the bounding rectangle 
    x, y, w, h = cv2.boundingRect(stats[i])

    # draw the field contour into the mask
    cv2.drawContours(mask, [stats[i]], -1, 0, -1)

    # remove the contour from the original image
    section = cv2.bitwise_and(section, section, mask=mask)

    # crop the boxed field
    field = crop_by_origin(x, y, w, h, padding=p)

This is what I get:

img

I don't understand why it didn't work? Maybe it's because in the site his example is on a black background? Maybe it doesn't work with "transparent" contours? Is this even possible?

How to fix this? Any other possible solutions around here?

UPDATE

I tried another image with @nathancy's answer and this is what came up:

img

The result:

img

I tried playing around with the horizontal line kernel but it didn't work out as intended, is there a way to make this more dynamic?

computer-vision opencv image-processing image python
1 Answers

This problem can be broken down into two separate steps. First, we want to isolate the rectangles which can be done using contour approximation + filtering. Then we remove the horizontal and vertical lines with the implementation borrowed from a previous answer in removing horizontal lines in image. Here's an overall approach:

  • Convert image to grayscale and Gaussian blur
  • Otsu's threshold to get binary image

  • Find contours and use contour approximation to find rectangle boxes

    • If a contour passes this filter and minimum threshold area, use Numpy slicing to copy-paste the ROI onto a blank white mask

  • Remove horizontal lines

  • Remove vertical lines
  • Bitwise-and mask with input image to get result

Otsu's Threshold -> Detected boxes -> Boxes drawn onto mask -> Inverted

Remove horizontal lines -> Remove vertical lines -> Bitwise-and to get result

import cv2
import numpy as np

image = cv2.imread('1.png')
mask = np.ones(image.shape, dtype=np.uint8) * 255
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (7,7), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    area = cv2.contourArea(c)
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.05 * peri, True)
    if len(approx) == 4 and area > 500:
        x,y,w,h = cv2.boundingRect(approx)
        mask[y:y+h, x:x+w] = image[y:y+h, x:x+w]

# Remove horizontal lines
mask = cv2.cvtColor(255 - mask, cv2.COLOR_BGR2GRAY)
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (50,2))
detect_horizontal = cv2.morphologyEx(mask, cv2.MORPH_OPEN, horizontal_kernel, iterations=1)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(mask, [c], -1, (0,0,0), 6)

# Remove vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2,45))
detect_vertical = cv2.morphologyEx(mask, cv2.MORPH_OPEN, vertical_kernel, iterations=1)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(mask, [c], -1, (0,0,0), 6)

# Bitwise mask with input image
result = cv2.bitwise_and(image, image, mask=mask)
result[mask==0] = (255,255,255)

cv2.imshow('mask', mask)
cv2.imshow('thresh', thresh)
cv2.imshow('result', result)
cv2.waitKey()
user_111621650
Related