Having thought this might be a fairly easy task, I wanted to take a screenshot of a unix terminal and convert it into text or as close to as if i had copied that text from the terminal. I have been digging around and the common choice for image to text seems to be cv2 to clean up and identify the relevant area's then tesseract for the conversion of identified areas to text. The below code was from various online sources mashed together but I have to say i don't fully know what the functions are all doing or how to achieve my expected output.
The code just identifies the entire screenshot as one countour to give to tesseract, its processing seems to process it in columns rather than lines.
Is there away where i can draw rectangles around each line and thus pass line by line to tesseract?
**UPDATE: **I have had more success setting the tesseact config to "--psm 6". However some of the letters are not quite detected correctly. Is there anything more I need to do to clean up the image?
# Import required packages
import cv2
import pytesseract
pytesseract.pytesseract.tesseract_cmd = r'C:\Users\cdoyle5\AppData\Local\Tesseract-OCR\tesseract.exe'
img = cv2.imread("term1.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh1 = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU | cv2.THRESH_BINARY_INV)
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (20, 20))
dilation = cv2.dilate(thresh1, rect_kernel, iterations=1)
contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
im2 = img.copy()
with open("text_from_terminal.txt", "w+") as file:
for cnt in contours:
x, y, w, h = cv2.boundingRect(cnt)
cv2.rectangle(im2, (x, y), (x + w, y + h), (0, 255, 0), 2)
# Cropping the text block for giving input to OCR
cropped = im2[y:y + h, x:x + w]
cv2.imshow("cropped", cropped)
cv2.waitKey(0)
text = pytesseract.image_to_string(cropped)
print("detected: ", text)
# Appending the text into file
file.write(text + "\n")
Expected output
ubuntu@vps-f116ed9f:~$ ls -lrt
total 538160
-rw-rw-r-- 1 ubuntu ubuntu 508206724 Oct 17 07:23 splunk-8.1.0-f57c09e87251-Linux-x86_64.tgz
drwxr-xr-x 3 ubuntu ubuntu 4096 Nov 1 21:17 snap
-rw-rw-r-- 1 ubuntu ubuntu 42847944 Nov 17 09:56 splunkforwarder-8.1.0-f57c09e87251-linux-2.6-x86_64.rpm
-rw-rw-r-- 1 ubuntu ubuntu 0 Nov 17 20:10 test
drwxrwxr-x 3 ubuntu ubuntu 4096 Dec 2 11:56 TM129VCE
-rw-rw-r-- 1 ubuntu ubuntu 226 Dec 8 16:34 Dockerfile
drwxrwxr-x 2 ubuntu ubuntu 4096 Dec 28 23:27 screen_to_text
Actual Output
ubuntu@v'ps—f116ed9f: 5.: ls —1rt
total 5 3 B 1 6 0
—rw—rw—r——
drwxrixrix
—rw—rw—r——
—rw—rw—r——
drwxrwxrix
—rw—rw—r——
drwxrwxrix
~>—Aw>—‘>—Aw>—A
ubuntu
ubuntu
ubuntu
ubuntu
ubuntu
ubuntu
ubuntu
ubuntu 503206724
ubuntu 4 0 9 6
ubuntu 42B47944
ubuntu 0
ubuntu 4 0 9 6
ubuntu 2 2 6
4 0 9 6
ubuntu
Oct
NOV
NOV
NOV
Dec
Dec
Dec
17
17
17
2B
07:23
21:17
09:56
20:10
11:56
16:34
23:27
sp1unk—a.1.0—f57c09ea7251—Linux—xa6_64.tgz
Snap
sp1unkforwarder—a.1.0—f57c09ea7251—1inux—2.6—xa6_64.rpm
test
TM129VCE
Dockerfile
screen_to_text
