ARKit - 2D Projection Result for a Point in 3D Space Changes Abruptly at Certain Device Position/Orientation

Viewed 108

I'm working on an ARKit app that uses SceneKit as the rendering engine. It does some typical ARKit things like allowing the user to place 3D virtual content in the scene, detect and draw horizontal and vertical planes, etc.

The app also includes a feature to create measurement objects like those found in Apple's Measure app. The user taps an "add endpoint" button to add the first point of the measurement, moves the device to target the location for the second, and then completes the measurement by tapping the "add endpoint" button again. Measurement Working

The issue is that with certain device orientations and distances from the measurement objects, the measurement objects behave erratically. The first end point will suddenly change position and the line and first end point will appear to "flip" about the second endpoint. Measurement "Flipping"

The measurement feature creates SpriteKit assets and adds them to the overlaySKScene of the sceneView. These measurements record their start and end point positions in 3D space and project these points to 2D to reposition their SpriteKit components.

This is how I'm currently updating the position of the measurement items:

func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
    measurementNodes.forEach { $0.updateComponentPositions(renderer: sceneView) }
    //...
}

The method updateComponentPositions is called every frame using the SCNSceneRendererDelegate method renderer(_, updateAtTime).

func updateComponentPositions(renderer: SCNSceneRenderer) {
    // update start circle
    let projectedStartPointScreenPosition = renderer.projectPoint(startPointPosition)
    let startPointSKPosition = CGPoint(x: CGFloat(projectedStartPointScreenPosition.x),
                                       y: scene.size.height - CGFloat(projectedStartPointScreenPosition.y) - startPointNode.frame.height/2)
    startPointNode.position = startPointSKPosition

    // update end circle
    let projectedEndPointScreenPosition = renderer.projectPoint(endPointPosition)
    let endPointSKPosition = CGPoint(x: CGFloat(projectedEndPointScreenPosition.x), y: scene.size.height - CGFloat(projectedEndPointScreenPosition.y) - endPointNode.frame.height/2)
    endPointNode.position = endPointSKPosition

    // update label
    labelBackgroundNode.position = CGPoint(x: (startPointSKPosition.x + endPointSKPosition.x) / 2,
                                           y: (startPointSKPosition.y + endPointSKPosition.y) / 2)

    // reset line path
    lineNode.path = getLinePath(from: startPointSKPosition,
                                to: endPointSKPosition)
}

This is the updateComponentPositions method of MeasurementNode which takes a renderer as an argument, projects the 3D points to 2D locations, and then updates the position of its geometries.

When attempting to debug this problem, I focused on the result of renderer.projectPoint(startPointPosition). The startPointPosition is a static SCNVector3 that holds the 3D position when the first endpoint was created. However, the result of this projectPoint method is what becomes erratic at this specific device orientation. Here is a sample of the log of renderer.projectPoint(startPointPosition) when I hit that "flipping" point with the device:

SCNVector3(x: -6692.817, y: -13238.885, z: 0.49546456)
SCNVector3(x: -8706.296, y: -17154.777, z: 0.49417993)
SCNVector3(x: -12774.009, y: -25042.207, z: 0.491588)
SCNVector3(x: -24064.623, y: -46880.246, z: 0.48439798)
SCNVector3(x: -164018.86, y: -317208.25, z: 0.39538962) // magnitude becomes very large for x and y
SCNVector3(x: 17587.36, y: 33517.477, z: 0.510834) // x and y "flip"
SCNVector3(x: 9163.234, y: 17199.012, z: 0.50547314)
SCNVector3(x: 7649.4766, y: 14249.298, z: 0.5045147)
SCNVector3(x: 6688.1143, y: 12365.713, z: 0.5039065)

I tried replacing the updateComponentPositions method with one that takes the ARCamera as an argument in order to see if I would have better luck with its projectPoint(_ point: simd_float3, orientation: UIInterfaceOrientation, viewportSize: CGSize) method. I called it in that same renderer(_, updateAtTime) method, but achieved the same result.

0 Answers
Related