RealityKit – Image recognition and working with many scenes

user_4506724 0
Viewed 763

I've created an app using the RealityKit template file. Inside RealityComposer there are multiple scenes, all the scenes use image recognition that activates some animations.

Inside Xcode I have to load all the scenes as anchors and append those anchors to arView.scene.anchors array. The issue is an obvious one, as I present the physical 2D image one after the other I get multiple anchors piled on top of each other which is not desirable. I'm aware of arView.scene.anchors.removeAll() prior to loading the new anchor but my issue is this:

How do I check when a certain image has appeared to therefore remove the existing anchor and load the correct one? I've tried to look for something like there is in ARKit as didUpdate but I can't see anything similar in RealityKit.

Many thanks

reality-composer swiftui realitykit arkit swift
1 Answers

Foreword

RealityKit's AnchorEntity(.image) coming from RC, matches ARKit's ARImageTrackingConfig. When iOS device recognises a reference image, it creates Image Anchor (that conforms to ARTrackable protocol) that tethers a corresponding 3D model. And, as you understand, you must show just one reference image at a time (in your particular case AR app can't operate normally when you give it two or more images simultaneously).


Code snippet showing how if condition logic might look like:

import SwiftUI
import RealityKit

struct ContentView : View {
    var body: some View {            
        return ARViewContainer().edgesIgnoringSafeArea(.all)
    }
}

struct ARViewContainer: UIViewRepresentable {

    func makeUIView(context: Context) -> ARView {    
        let arView = ARView(frame: .zero)

        let id02Scene = try! Experience.loadID2()            
        print(id02Scene)     // prints scene hierarchy
        
        let anchor = id02Scene.children[0]
        print(anchor.components[AnchoringComponent] as Any)
        
        if anchor.components[AnchoringComponent] == AnchoringComponent(
                      .image(group: "Experience.reality", 
                              name: "assets/MainID_4b51de84.jpeg")) {
            
            arView.scene.anchors.removeAll()
            print("LOAD SCENE")
            arView.scene.anchors.append(id02Scene)
        }
        return arView
    }
 
    func updateUIView(_ uiView: ARView, context: Context) { }
}

ID2 scene hierarchy printed in console:

enter image description here

P.S.

You should implement SwiftUI Coordinator class (read about it here), and inside Coordinator use ARSessionDelegate's session(_:didUpdate:) instance method to update anchors properties at 60 fps.

Also you may use the following logic – if anchor of scene 1 is active or anchor of scene 3 is active, just delete all anchors from collection and load scene 2.

var arView = ARView(frame: .zero)

let id01Scene = try! Experience.loadID1()
let id02Scene = try! Experience.loadID2()
let id03Scene = try! Experience.loadID3()

func makeUIView(context: Context) -> ARView {    
    arView.session.delegate = context.coordinator

    arView.scene.anchors.append(id01Scene)
    arView.scene.anchors.append(id02Scene)
    arView.scene.anchors.append(id03Scene)
    return arView
}

...

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    if arView.scene.anchors[0].isActive || arView.scene.anchors[2].isActive {
        arView.scene.anchors.removeAll()
        arView.scene.anchors.append(id02Scene)
        print("Load Scene Two")
    }
}
user_65995900
Related