Basically I'm trying to render a user with the front camera of an iPhone 10+ device using the depth mask and composite them on top of a 3D SceneKit scene. The trick is I'm trying to make things as accurate as they are when utilizing TrueDepth. I'd like to be able to still record video. Access to Apple's camera stream is done via a AVCaptureSession. This is used both by the depth map extraction code and ARKit’s ARSession API.
However, when trying to connect my depth map effort to my occlusion prototype, I am blocked by the operating system. Either I could get depth information from the camera, or I could have an ARSession running. But not both at the same time. Could I get around this by using the depth data delivered as part of ARFrame as an input to my occlusion/rendering system? And if so, how would I go about there or most importantly where could I go to read more on how to do that?
Apologies if the question is convoluted. I haven't slept in days trying to figure this problem out.
Please let me know if I'm not including enough information or need to clarify.