AVMetaDataObject.bounds to SwiftUI position?

Viewed 116

I have two views. The parent view holds a CameraFeed UIViewControllerRepresentable which passes back the bounds of a AVMetadataFaceObject. I'm trying to draw overlays where that bounds should be. I'm getting kind of close, but my mappings aren't quite right. The CameraFeed passes back a scalar-style set of bounds and I'm multiplying my geometry reader by it.

  • The X and the Y seem to be swapped? (orange square works wrong and blue square works better)
  • The location doesn't QUITE line up perfectly.

Acknowledgements: Thanks to the HackingWithSwift InstaFilter and Hot Prospects examples, btw. They were very helpful to getting this far.

I do understand that there is a CALayer that I could draw on in the AV object, and if I went that way I could use the more powerful Vision framework but I was seeing if I could try this approach first. I also am aware that there is a CIFaceFeature I could be using too. Also, I don't have a TrueDepth front-facing camera to work with. I just wanted to see if I could hijack this seemingly simplest of solutions to make it work.

What am I missing about the how how the bounds of AVMetaDataObject work and approaches to doing the frame of reference transformation? Thanks in advance. Full Code on GitHub

struct CameraFeedView: View {
    @State var foundFace:CGRect?
    @State var geometryRect:CGRect = CGRect(x: 0, y: 0, width: 0, height: 0)
    //@State var foundFaceAdjusted:CGRect?
    //var testRect:CGRect = CGRect(
    //           x: 0.44112000039964916, 
    //           y: 0.1979580322805941, 
    //           width: 0.3337599992007017, 
    //           height: 0.5941303606941507)
    
    var body: some View {
        GeometryReader(content: { geometry in
            ZStack {
                CameraFeed(codeTypes: [.face], completion: handleCameraReturn)
                if (foundFace != nil) {
                    Rectangle()
                        .stroke()
                        .foregroundColor(.orange)
                        .frame(width: 100, height: 100, alignment: .topLeading)
                        .position(
                            x: geometry.size.width * foundFace!.origin.x, 
                            y: geometry.size.height * foundFace!.origin.y)
                        
                    FoundObject(frameRect: geometryRect, boundsRect: foundFace!)
                        .stroke()
                        .foregroundColor(.blue)
                }
            }
            .onAppear(perform: {
                let frame = geometry.frame(in: .global)
                geometryRect = CGRect(
                                origin: CGPoint(x: frame.minX, y: frame.minY), 
                                size: geometry.size
                               )
            })
        })
        
    }
    
    func handleCameraReturn(result: Result<CGRect, CameraFeed.CameraError>) {
        switch result {
        case .success(let bounds):
            print(bounds)
            foundFace = bounds
            //TODO: Add a timer
        case .failure(let error):
            print("Scanning failed: \(error)")
            foundFace = nil
        }
    }

}

struct FoundObject: Shape {
    func reMapBoundries(frameRect:CGRect, boundsRect:CGRect) -> CGRect {
        //Y bounded to width? Really?
        let newY = (frameRect.width * boundsRect.origin.x) + (1.0-frameRect.origin.x)
        //X bounded to height? Really?
        let newX = (frameRect.height * boundsRect.origin.y) + (1.0-frameRect.origin.y)
        let newWidth = 100//(frameRect.width * boundsRect.width)
        let newHeight = 100//(frameRect.height * boundsRect.height)
        let newRect = CGRect(
              origin: CGPoint(x: newX, y: newY), 
              size: CGSize(width: newWidth, height: newHeight))
        return newRect
    }
    
    let frameRect:CGRect
    let boundsRect:CGRect
    func path(in rect: CGRect) -> Path {
        var path = Path()
        path.addRect(reMapBoundries(frameRect: frameRect, boundsRect: boundsRect))
        return path
    }
}
0 Answers
Related