I am doing some deep learning on forms that have been scanned, but for a variety of reasons (poor scanner, creased paper, etc), I have an issue with the sections of the page being wonky.
Each section of the form is enclosed by rectangles, and I have a model that can identify and crop out the individual sections, but they are still warped, and warped in ways that a matrix transformations alone can't fix, which creates issues for what I want to do. Is there an algorithm that works with rectangles to re-align an image?
Here is an illustration of what I would need as inputs and outputs.
I have tried using ORB to align the images to a template, which works about 90% of the time, but the other 10% of the time I get a mangled mess of an image that is totally unusable.