How to prevent INPAINTING blocks and improve coloring

Question

I wanted to Remove all the texts USING INPAINTING from this IMAGE. I had been trying various methods, and eventually found that I can get the results through OCR and then using thresholding MASK THE IMAGE.

processedImage = preprocess(partOFimg) 
mask = np.ones(img.shape[:2], dtype="uint8") * 255
for c in cnts:
        cv2.drawContours(mask, [c], -1, 0, -1)
img = cv2.inpaint(img,mask,7,cv2.INPAINT_TELEA)

Preprocess operations:

  ret,thresh1 = cv2.threshold(gray, 0, 255,cv2.THRESH_OTSU|cv2.THRESH_BINARY_INV)
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15, 3))
dilation = cv2.dilate(thresh1, rect_kernel, iterations = 1)
edged = cv2.Canny(dilation, 50, 100)
cnts = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
mask =

np.ones(img.shape[:2], dtype="uint8") * 255

When I run the above code, I here am the OUTPUT Image OUTPUT. As we can see, it is making some BLOCKS OF DIFFERENT COLOR over the IMAGE, I want to prevent that, How do I achieve this? I see that mask images are not formed well many times, and in cases when the text is white the PREPROCESSING doesn't occur properly. How do I prevent these BLOCKS of other colours to FORM on the IMAGE?

Grayed Sub Image GRAYED

Threshold Sub IMG part: Thresholded Image

Masked Image Masked

EDIT 1:

I've managed to get this new better result by noticing that my threshold is the best mask I can get. After doing this I performed the masking process 3 different times with variable masks and inversions. I did the inpainting algorithm 3 times, it basically the other times inverse the mask, because in some cases required mask is the inversed mask. Still I think it needs improvement, If I chose a different image the results are not so good.

fmw42 · Answer

Python/OpenCV inpaint methods, generally, are not appropriate to your type of image. They work best on thin (scratch-like) regions, not large blocks. You really need an exemplar type method such as https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/criminisi_tip2004.pdf. But OpenCV does not have that.

However, the OpenCV methods do work here, I suspect, because you are filling with constant colors (green) and not texture. So you are best to try to get the mask of just the letters (characters), not rectangular blocks for the words. So, to show you what I mean, here is my Python/OpenCV approach.

Input:

Read the input
Threshold on the green sign
Apply morphology to close it up and keep as mask1
Apply the mask to the image to blacken out the outside of the sign
Threshold on the white in this new image and keep as mask2
Apply morphology dilate to enlarge it slightly and save as mask3
Do the inpaint
Save the results

import cv2
import numpy as np

# read input
img = cv2.imread('airport_sign.jpg')

# threshold on green sign
lower = (30,80,0)
upper = (70,120,20)
thresh = cv2.inRange(img, lower, upper)

# apply morphology close
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (135,135))
mask1 = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# apply mask to img
img2 = img.copy()
img2[mask1==0] = (0,0,0)

# threshold on white
#gray = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
#mask2 = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
lower = (120,120,120)
upper = (255,255,255)
mask2 = cv2.inRange(img2, lower, upper)

# apply morphology dilate
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
mask3 = cv2.morphologyEx(mask2, cv2.MORPH_DILATE, kernel)


# do inpainting
result1 = cv2.inpaint(img,mask3,11,cv2.INPAINT_TELEA)
result2 = cv2.inpaint(img,mask3,11,cv2.INPAINT_NS)

# save results
cv2.imwrite('airport_sign_mask.png', mask3)
cv2.imwrite('airport_sign_inpainted1.png', result1)
cv2.imwrite('airport_sign_inpainted2.png', result1)

# show results
cv2.imshow('thresh',thresh)
cv2.imshow('mask1',mask1)
cv2.imshow('img2',img2)
cv2.imshow('mask2',mask2)
cv2.imshow('mask3',mask3)
cv2.imshow('result1',result1)
cv2.imshow('result2',result2)
cv2.waitKey(0)
cv2.destroyAllWindows()

Mask 3:

Inpaint 1 (Telea):

Inpaint 2 (NS):

How to prevent INPAINTING blocks and improve coloring

1 Answers