Turn occlusion on for ARFaceTrackingConfiguration

Question

I am trying to follow the steps to create this following this article (image below from article):

It is basically recognising a face to put something on the face (a tattoo) and placing a background image behind it.

I am using an iPhone X device to test the code, but every time I test if .personSegmentation is supported, it is false:

if ARFaceTrackingConfiguration.supportsFrameSemantics(.personSegmentation) {
    configuration.frameSemantics.insert(.personSegmentation) // code never executed.
}

My whole code for adding the plane to put on top of the face plus the background is:

The ARSCNViewDelegate delegate to add the nodes:

private var virtualBackgroundNode = SCNNode()
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
    guard let device = sceneView.device else {
        return nil        
    }

    let faceGeometry = ARSCNFaceGeometry(device: device)
    let faceNode = SCNNode(geometry: faceGeometry)
    faceNode.geometry?.firstMaterial?.transparency = 0

    let tattooPlane = SCNPlane(width: 0.13, height: 0.06)
    tattooPlane.firstMaterial?.diffuse.contents = UIImage(named: "Tattoos/tattoo0")!
    tattooPlane.firstMaterial?.isDoubleSided = true

    let tattooNode = SCNNode()
    tattooNode.position.z = faceNode.boundingBox.max.z * 3 / 4
    tattooNode.position.y = 0.027
    tattooNode.geometry = tattooPlane

    faceNode.addChildNode(tattooNode)
    configureBackgroundView()
    sceneView.scene.rootNode.addChildNode(virtualBackgroundNode)

    return faceNode
}

Resizing the background image, setting it and positioning the background node:

func configureBackgroundView() {
    let (skScene, mediaAspectRatio) = makeImageBackgroundScene(image: UIImage(named: "Cats/cat0")!)
    let size = skScene.size
    virtualBackgroundNode.geometry = SCNGeometry.Plane(width: size.width, height: size.height)
    let material = SCNMaterial()
    material.diffuse.contents = skScene
    virtualBackgroundNode.geometry?.materials = [material]
    virtualBackgroundNode.scale = SCNVector3(1.7  * mediaAspectRatio, 1.7, 1)
    let cameraPosition = sceneView.pointOfView?.scale
    let position = SCNVector3(cameraPosition!.x, cameraPosition!.y, cameraPosition!.z - 1000)
    virtualBackgroundNode.position = position   
}

This method creates a SpriteKit image by resizing the original asset:

func makeImageBackgroundScene(image: UIImage) -> (scene: SKScene, mediaAspectRatio: Double) {
    //Adjusted so that the aspect ratio of the image is not distorted
    let width = image.size.width
    let height = image.size.height
    let mediaAspectRatio = Double(width / height)
    let cgImage = image.cgImage!
    let newImage = UIImage(cgImage: cgImage, scale: 1.0, orientation: .up)

    let skScene = SKScene(size: CGSize(width: 1000  * mediaAspectRatio, height: 1000))
    let texture = SKTexture(image: newImage)
    let skNode = SKSpriteNode(texture:texture)
    skNode.position = CGPoint(x: skScene.size.width / 2.0, y: skScene.size.height / 2.0)
    skNode.size = skScene.size
    skNode.yScale = -1.0
    skScene.addChild(skNode)

    return (skScene, mediaAspectRatio)
}

Any advice on what to try? Snapchat and TikTok have similar Face recognition + background setups and they work in my device.

Thanks for any help.

Turn occlusion on for ARFaceTrackingConfiguration

0 Answers