How to detect checkboxes by removing noise using Python OpenCV?

user_15734920 0
Viewed 111

I am trying to identify the checkboxes in the image

The top 4 are identified but the bottom 2 are not. At the same time I would like to be able to get rid of the peppering to avoid false positives as there are other docs that have checkmarks that are much smaller. I've tried various dilation and kernel sizes but I haven't been able to successful get the box.

I've tried to dilate it and then erode it

kernel = np.ones((2, 2), np.uint8)
image_dilat = cv2.dilate(image, kernel, iterations=1)
kernel = np.ones((4, 4), np.uint8)
image_erosion = cv2.erode(image_dilat2, kernel, iterations=1)

I've tried morphing it as well

kernel = np.ones((3, 3), np.uint8)
image = cv2.morphologyEx(image, cv2.MORPH_OPEN, kernel, iterations=1)

kernel = np.ones((3, 3), np.uint8)
image = cv2.morphologyEx(image, cv2.cv2.MORPH_CLOSE, , kernel, iterations=1)

Any suggestion will be appreciated.

computer-vision opencv image-processing image python
1 Answers

Here's a potential approach using simple image processing:

  1. Obtain binary image. Load the image, convert to grayscale, and Otsu's threshold.

  2. Remove small pixels of noise. Find contours and filter out noise using contour area filtering. We effectively remove the noise by "drawing in" the contours with black.

  3. Repair checkbox walls. From here we create a horizontal and vertical repair kernel then perform morphological close to fix any holes in the checkbox walls.

  4. Detect checkboxes. Next find contours on the repaired image then filter for checkbox contours using shape approximation and aspect ratio filtering. The idea is that a checkbox is a square and should have roughly the same width and height.

Binary image with noise -> Removed tiny noise

Repaired checkbox walls -> Detected checkboxes

Code

import cv2

# Load image, convert to grayscale, Otsu's threshold
image = cv2.imread('1.png')
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
cv2.imshow('thresh before', thresh)

# Find contours and filter using contour area filtering to remove noise
cnts, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[-2:]
AREA_THRESHOLD = 10
for c in cnts:
    area = cv2.contourArea(c)
    if area < AREA_THRESHOLD:
        cv2.drawContours(thresh, [c], -1, 0, -1)

# Repair checkbox horizontal and vertical walls
repair_kernel1 = cv2.getStructuringElement(cv2.MORPH_RECT, (5,1))
repair = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, repair_kernel1, iterations=1)
repair_kernel2 = cv2.getStructuringElement(cv2.MORPH_RECT, (1,5))
repair = cv2.morphologyEx(repair, cv2.MORPH_CLOSE, repair_kernel2, iterations=1)

# Detect checkboxes using shape approximation and aspect ratio filtering
cnts, _ = cv2.findContours(repair, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2:]
for c in cnts:
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.05 * peri, True)
    x,y,w,h = cv2.boundingRect(approx)
    aspect_ratio = w / float(h)
    if aspect_ratio > 0.9 and aspect_ratio < 1.1:
        cv2.rectangle(original, (x, y), (x + w, y + h), (36,255,12), 3)

cv2.imshow('thresh', thresh)
cv2.imshow('repair', repair)
cv2.imshow('original', original)
cv2.waitKey()

Note: The assumption is that the checkboxes are square shaped and that there are no noise overlapping the checkboxes. Depending on the image, you may want to add another layer of contour area filtering to ensure that you don't get false positives.

user_111621650
Related