CapturedImage in ARFrame to portrait/vertical

There's an API for that.

ARFrame.displayTransform(for:viewportSize:) gives you a transform matrix that rotates from the image's sensor-native orientation to whatever UI orientation you specify, and provides a scale factor to fit a viewport whose aspect ratio doesn't necessarily match that of the camera.

You can then use that transform in whatever display pipeline you're working with to show the rotated image. For example:

• check the Xcode project template for a Metal-based ARKit app to see how they feed that transform matrix to a vertex shader.

• create a Core Image CIImage from the pixel buffer, then use transformed(by:) to get a rotated image, which you can then convert to UIImage and display in UIImageView.

frame.displayTransform(for: .portrait, viewportSize: CGSize(width: height, height: width)).inverted() rotated my portrait CGRect to the landscape orientation that ARFrame captures images with.