Does anyone know how to reproduce the new Notes new scanning feature in iOS 11??
Is AVFoundation used for the camera?
How is the camera detecting the shape of the paper/document/card?
How do they place the overlay over in real time?
How does the camera know when to take the photo?
What's that animated overlay and how can we achieve this?
Does anyone know how to reproduce this? Not exactly :P
Is AVFoundation used for the camera? Yes
How is the camera detecting the shape of the paper/document/card? They are using the Vision Framework to do rectangle detection. It's stated in this WWDC session by one of the demonstrators
How do they place the overlay over in real time? You Should check out the above video for this as he talks about doing something similar in one of the demos
How does the camera know when to take the photo? I'm not familiar with this app but it's surely triggered in the capture session, no?
Whats that animated overlay and how can we achieve this? Not sure about this but I'd imagine it's some kind of CALayer with animation
Is Tesseract framework used for the image afterwards? Isn't Tesseract OCR for text? If you're looking for handwriting recognition, you might want to look for a MNIST model
Use Apple’s rectangle detection SDK, which provides an easy-to-use API that can identify rectangles in still images or video sequences in near-realtime. The algorithm works very well in simple scenes with a single prominent rectangle in a clean background, but is less accurate in more complicated scenes, such as capturing small receipts or business cards in cluttered backgrounds, which are essential use-cases for our scanning feature.
An image processor that identifies notable features (such as faces and barcodes) in a still image or video.
https://developer.apple.com/documentation/coreimage/cidetector
