What is the difference between sceneView.pointOfView.transform and ARCamera.transform in ARKit?

user_3427580 0
Viewed 279

I have a fair background with camera geometry, but I am new to Swift and ARKit development, and I am getting a bit confused between the different types of transform matrices that can be returned.

In particular, one tutorial I followed had me grabbing the ARSCNView as sceneView and then printing sceneView.pointOfView.transform. This showed me what I expect: when the session is begun, the 4x4 matrix is very close to identity.

However, when using the ARCamera associated with each ARFrame, as in this code snippet:

extension ViewController: ARSessionDelegate {
    
    func session(_ session: ARSession, didUpdate frame: ARFrame) {
        print(frame.camera.transform.transpose)
    }
}

I instead get a camera matrix that looks like (note the transpose is on purpose as I am more familiar with the row-major layout):

[0  1  0  0]
[-1 0  0  0]
[0  0  1  0]
[0  0  0  1]

I've tried explicitly the ARWorldAlignment property to .gravity and .gravityAndHeading, but neither produces the expected identity matrix when the session is run.

I also added the .showWorldOrigin debug option, which shows the world axes as I would expect: +X to the right of my phone, +Y up, and -Z down the view axis.

What am I missing here? Can someone please clarify the coordinate systems of the transforms that ARFrame is returning and how they differ from those of ARSCNView?

scenekit realitykit arkit augmented-reality swift
1 Answers

The only difference is in types. Identity matrices are the same in any case.

However, the inverse of a rotation matrix is its transpose.


SCNMatrix4

// SCNMatrix4 is a typealias of CATransform3D

@IBOutlet var sceneView: ARSCNView!

sceneView.pointOfView?.transform.m41          // camera position x
sceneView.pointOfView?.transform.m42          // camera position y
sceneView.pointOfView?.transform.m43          // camera position z

┌                      ┐
|  m11  m21  m31  m41  |
|  m12  m22  m32  m42  |
|  m13  m23  m33  m43  |
|  m14  m24  m34  m44  |
└                      ┘

float4x4 | simd_float4x4

@IBOutlet var arView: ARView!
    
arView.cameraTransform.matrix.columns.3.x     // camera position x
arView.cameraTransform.matrix.columns.3.y     // camera position y
arView.cameraTransform.matrix.columns.3.z     // camera position z

// 0  1  2  3  indices 
┌              ┐
|  1  0  0  x  |
|  0  1  0  y  |
|  0  0  1  z  |
|  0  0  0  1  |
└              ┘

simd_float4x4

extension ViewController: ARSessionDelegate {

    func session(_ session: ARSession, didUpdate frame: ARFrame) {
    
        frame.camera.transform.columns.3.x    // camera position x
        frame.camera.transform.columns.3.y    // camera position y
        frame.camera.transform.columns.3.z    // camera position z
    }
}

┌              ┐
|  1  0  0  x  |
|  0  1  0  y  |
|  0  0  1  z  |
|  0  0  0  1  |
└              ┘

However if you print frame.camera.transform you'll get a horizontal representation of columns:

//   1st column     2nd column     3rd column     4th column
    ([1, 0, 0, 0],  [0, 1, 0, 0],  [0, 0, 1, 0],  [x, y, z, 1])
user_65995900
Related