How to reduce space between letters and fix rotated letters in this CAPTCHA

user16514821 0
Viewed 137

I am trying to get the text from rotated letters from images by using the Python cv2 and tesseract ocr library but am unable to do that.

Input Image:

Input Image

My first approach to split into multiple letters and fix the rotate letter's and then extract the text which didn't works.

import cv2
from imutils import contours

image = cv2.imread('input_0.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray,0,255,cv2.THRESH_OTSU + cv2.THRESH_BINARY)[1]

cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts, _ = contours.sort_contours(cnts, method="left-to-right")

ROI_number = 0
for c in cnts:
    area = cv2.contourArea(c)
    if area > 10:
        x,y,w,h = cv2.boundingRect(c)
        ROI = 255 - image[y:y+h, x:x+w]
        cv2.imwrite('ROI_{}.png'.format(ROI_number), ROI)
        cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 1)
        ROI_number += 1
cv2.imshow('thresh', thresh)
cv2.imshow('image', image)
cv2.waitKey(0)

Output Getting:

output generated by above code

Now

I want to reduce the space between the letters and fix the rotated letters in the image but I am not getting any solution for the same.

UPDATE

Tried to rotate the letter by using below code but am unable

cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, 
cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
rect = cv2.minAreaRect(cnts[0])
box = np.int0(cv2.boxPoints(rect))
cv2.drawContours(image, [box], 0, (36,255,12), 3) # OR
# cv2.polylines(image, [box], True, (36,255,12), 3)

cv2.imshow('image', image)
cv2.waitKey()

Expected Output

784FIK

Please suggest how to reduce the space between the letters and fix the rotated letters? then may be tesseract ocr works.

captcha tesseract ocr opencv python
1 Answers

I managed to make some progress, though it seems to be unstable to me. You need to check with larger datasets.

image = cv2.imread('input_0.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray,0,255,cv2.THRESH_OTSU + cv2.THRESH_BINARY)[1]
thresh = 255 -thresh

cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts, _ = contours.sort_contours(cnts, method="left-to-right")

ROI_number = 0
offset = 6
custom_oem_psm_config = r'--oem 3 --psm 10' # psm is crucial for the detection
# -> https://pyimagesearch.com/2021/11/15/tesseract-page-segmentation-modes-psms-explained-how-to-improve-your-ocr-accuracy/

for c in cnts:
    area = cv2.contourArea(c)
    if area > 10:
        x,y,w,h = cv2.boundingRect(c)
        ROI = image[y-offset:y+h+offset, x-offset:x+w+offset]
        
        angle = cv2.minAreaRect(c)[-1]

        # letters need to be rotated and placed into an image
        # with larger area to have enough space around the letters

        # I did this manually in ImageJ/Fiji, but it should also be possible in opencv.


        img_name = 'ROI_{}.png'.format(ROI_number)
        cv2.imwrite(img_name, ROI)
        print(pytesseract.image_to_string(ROI, config=custom_oem_psm_config))
        cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 1)
        ROI_number += 1

The angle of an individual character needs to be corrected to 0° or 90°. only the '4' needed to be rotated (result was 38°) by either nothing or towards 45°, I could not figure out why so far.

The image I generated with these angles + aligning the letters in ImageJ: imagej_characters.png

This

custom_oem_psm_config = r'--oem 3 --psm 10'
image = cv2.imread('imagej_characters.png')
print(pytesseract.image_to_string(image, config=custom_oem_psm_config))

was able to read 784FIK.

Hope this will help you to improve your results.

userOctl10
Related