Unix terminal screenshot to text

Viewed 277

Having thought this might be a fairly easy task, I wanted to take a screenshot of a unix terminal and convert it into text or as close to as if i had copied that text from the terminal. I have been digging around and the common choice for image to text seems to be cv2 to clean up and identify the relevant area's then tesseract for the conversion of identified areas to text. The below code was from various online sources mashed together but I have to say i don't fully know what the functions are all doing or how to achieve my expected output.

The code just identifies the entire screenshot as one countour to give to tesseract, its processing seems to process it in columns rather than lines.

Is there away where i can draw rectangles around each line and thus pass line by line to tesseract?

**UPDATE: **I have had more success setting the tesseact config to "--psm 6". However some of the letters are not quite detected correctly. Is there anything more I need to do to clean up the image?

# Import required packages
import cv2
import pytesseract

pytesseract.pytesseract.tesseract_cmd = r'C:\Users\cdoyle5\AppData\Local\Tesseract-OCR\tesseract.exe'
img = cv2.imread("term1.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh1 = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU | cv2.THRESH_BINARY_INV)
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (20, 20))
dilation = cv2.dilate(thresh1, rect_kernel, iterations=1)
contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
im2 = img.copy()

with open("text_from_terminal.txt", "w+") as file:
    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)

        cv2.rectangle(im2, (x, y), (x + w, y + h), (0, 255, 0), 2)
        # Cropping the text block for giving input to OCR
        cropped = im2[y:y + h, x:x + w]
        cv2.imshow("cropped", cropped)
        cv2.waitKey(0)
        text = pytesseract.image_to_string(cropped)
        print("detected: ", text)

        # Appending the text into file
        file.write(text + "\n")

Screenshot sample enter image description here

Expected output

ubuntu@vps-f116ed9f:~$ ls -lrt
total 538160
-rw-rw-r-- 1 ubuntu ubuntu 508206724 Oct 17 07:23 splunk-8.1.0-f57c09e87251-Linux-x86_64.tgz
drwxr-xr-x 3 ubuntu ubuntu      4096 Nov  1 21:17 snap
-rw-rw-r-- 1 ubuntu ubuntu  42847944 Nov 17 09:56 splunkforwarder-8.1.0-f57c09e87251-linux-2.6-x86_64.rpm
-rw-rw-r-- 1 ubuntu ubuntu         0 Nov 17 20:10 test
drwxrwxr-x 3 ubuntu ubuntu      4096 Dec  2 11:56 TM129VCE
-rw-rw-r-- 1 ubuntu ubuntu       226 Dec  8 16:34 Dockerfile
drwxrwxr-x 2 ubuntu ubuntu      4096 Dec 28 23:27 screen_to_text

Actual Output

ubuntu@v'ps—f116ed9f: 5.: ls —1rt

total 5 3 B 1 6 0
—rw—rw—r——
drwxrixrix
—rw—rw—r——
—rw—rw—r——
drwxrwxrix
—rw—rw—r——
drwxrwxrix

~>—Aw>—‘>—Aw>—A

ubuntu
ubuntu
ubuntu
ubuntu
ubuntu
ubuntu
ubuntu

ubuntu 503206724

ubuntu 4 0 9 6
ubuntu 42B47944
ubuntu 0
ubuntu 4 0 9 6
ubuntu 2 2 6

4 0 9 6

ubuntu

Oct
NOV
NOV
NOV
Dec
Dec
Dec

17

17
17

2B

07:23
21:17
09:56
20:10
11:56
16:34
23:27

sp1unk—a.1.0—f57c09ea7251—Linux—xa6_64.tgz

Snap
sp1unkforwarder—a.1.0—f57c09ea7251—1inux—2.6—xa6_64.rpm
test

TM129VCE

Dockerfile

screen_to_text


1 Answers

From what I understand, you want to save the output of command in "text_from_terminal.txt", if this is you want then you can simply do this by this code :-

import os

output = os.popen("ls -lrt").read()

with open("text_from_terminal.txt","w+") as file:
    file.write(output)
Related