I am new to the whole image processing and ocr topic.
The Task: What I am trying to do is to read in articles and the money paid for them from a receipt. The receipt is most likely a picture taken by a smart phone camera. I had a try with Tesseract ocr which worked OK but did not gave me all the text from the receipt. Especially the prices where missing. After that I figured I could use opencv to extract the necessary text areas first and feed only them to the Tesseract. That is what I need help with. It would be ideal if the approach would be reusable for different qualities and formats of the image. Please keep in mind that it is most important to have a clear association between article and price. The actual ocr will be supported by a database. Read “Additional Info” section for the big picture.
The Approach: My approach so far is to extract the receipt area from the picture as gray scale and further brake it down afterwards. Of corse the image is normalized to a fixed width and hight and I am trying around with blurs. I did manage to get horizontal line separation by converting the image to black and white, getting a histogram by reducing the image to one vector with the sum of all the pixel values in one horizontal line. That histogram is smoothed so that I can get clear local minima from it. That still left me with the problem to figure out which of those lines actually belong the articles on the receipt and furthermore I would need a similar thing for the vertical separation between article and price.
The Question: The first question would be: Am I even on the right track? Maybe Tesseract would do well for the whole image already if I did the right preprocessing? Or is there even a full solution available to what I am trying? Otherwise you may have some alternatives to calculation a histogram for line separation? Furthermore an Image of a receipt does pretty much never have straight lines. Is there an easy way (in the black and white picture) of having my line separators to be pushed in a white direction if it hits a black area (a letter) similar to using an opencv kernel along that line but hav it dynamically move up and down? The outcome would be a curved blue line truly separating two lines of text (I could program that thing on my own in python but it would be inefficient I think)
Additional Infos: What you may need to know is that this task is part of a bigger project where I like to have purchases written into a database for analytical purposes. Therefore my articles can always be matched against an existing set of articles or they can be corrected manually and added. Having the image scaled down for the processing and scaling the found areas up again to use the original image for the actual ocr is an option. There will be some kind of additional software surrounding the image processing. That means the images will be uploaded to a server (for example a Raspberry Pi) or ideally the processing will take place in a smart phone App later. Therefore low hardware resource consumption is preferred if possible.
Other things I tried are: Hough Lines to get boxes around text but I found no straight lines in the image. Fooling around with blur and sharpening seemed to not have that much of an impact in this stage of image processing. I only used blurring to erode the text on the receipt and have a pretty much white area which then is my receipt area. I also read about having areas calculated by minimizing the size and maximizing it while keeping pixel density (text) as high as possible but it seamed like a bit complicated for what I am trying to do?
I would love to provide additional information for a bedder understanding and many thanks in advance!
Images:
