How VNClassificationObservation identifier works in VNCoreMLRequest

Viewed 9

Im new in Core ML Model implementation. Im working on example of ARKit with Core ML. I have created core ML model with set of images. There are two folder, one folder (named as 'James') in which I have 3 images of my friend with front, left and right angle. and other folder (named as 'Unknown') qhwew I have some random faces. I kept iteration maximum 20 and augmented data options I have selected only crop.

I have following code where I'm integrating that Model. My requirement is when Im scanning the face of friend it should display my friend name as I mentioned in the Core Model and if any other face or any random face scanned then it should display "Uknown" or "This is not me" something text.

With this code when Im scanning my friend face its not showing his name as James or when Im scanning any random face apart from my friend face then its not showing Unknown as text. Where Im exactly lacking? What is the issue?

    
    let sceneView = ARSCNView(frame: UIScreen.main.bounds)
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        sceneView.delegate = self
        sceneView.showsStatistics = true
        guard ARFaceTrackingConfiguration.isSupported else { return }
        let configuration = ARFaceTrackingConfiguration()
        
        configuration.isLightEstimationEnabled = true
        sceneView.session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
        view.addSubview(sceneView)
    }
}

extension ViewController: ARSCNViewDelegate {
    
    func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
        
        guard let device = sceneView.device else {
            return nil
        }
        
        let faceGeometry = ARSCNFaceGeometry(device: device)
        
        let node = SCNNode(geometry: faceGeometry)
        
        node.geometry?.firstMaterial?.fillMode = .lines
        
        return node
    }
    
    func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
        
        guard let faceAnchor = anchor as? ARFaceAnchor,
            let faceGeometry = node.geometry as? ARSCNFaceGeometry else {
                return
        }
        
        faceGeometry.update(from: faceAnchor.geometry)
        
        let text = SCNText(string: "", extrusionDepth: 2)
        let font = UIFont(name: "Avenir-Heavy", size: 20)
        text.font = font
        let material = SCNMaterial()
        material.diffuse.contents = UIColor.green
        text.materials = [material]
        text.firstMaterial?.isDoubleSided = true
        
        let textNode = SCNNode(geometry: faceGeometry)
        textNode.position =    SCNVector3(-0.1, -0.1, -0.5)
        print(textNode.position)
        textNode.scale = SCNVector3(0.002, 0.002, 0.002)
        
        textNode.geometry = text
        
        guard let model = try? VNCoreMLModel(for: FaceRecognitionPerson_1().model) else {
            fatalError("Unable to load model")
        }
        
        let coreMlRequest = VNCoreMLRequest(model: model) {[weak self] request, error in
            guard let results = request.results as? [VNClassificationObservation],
                let topResult = results.first
                else {
                    fatalError("Unexpected results")
            }

            DispatchQueue.main.async {[weak self] in
                print("Identifier Received //: ===> \(topResult.identifier)")
                
                text.string = topResult.identifier
                if topResult.identifier != "James" {
                    print("**===Known User Detected**===")
                }
                self!.sceneView.scene.rootNode.addChildNode(textNode)
                self!.sceneView.autoenablesDefaultLighting = true
            }
        }
        
        guard let pixelBuffer = self.sceneView.session.currentFrame?.capturedImage else { return }
        
        let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
        DispatchQueue.global().async {
            do {
                try handler.perform([coreMlRequest])
            } catch {
                print(error)
            }
        }
    }
0 Answers
Related