Does SceneKit support 8-bit RGB colours for Point Clouds?

Viewed 191

Using Swift 5 and SceneKit on Xcode 12.5.1 and iOS 14.6

Trying to display a point cloud: I am trying to use 8-bit bytes for red/green/blue channels instead of 32-bit floats to save RAM (an iPhone 8 struggles with a mere 20 million point cloud; I must be using more transient RAM than necessary), but the function logs a non-fatal error message and does not use the colours provided.

It works fine with 32-bit floats. The error message is the same on the Simulator and the iPhone 8 - "[SceneKit] Error: unsupported conversion uchar4 -> float4"

I note that if I change it to one channel of 8-bits, I get "Warning: C3DBaseTypeFromDescription - signed/unsigned ambiguity" which according to a quick google no-one on the entire internet has seen (it gets two stack overflow hits which look promising but neither one has the actual word in it!) Bing thinks that word is in ads for massage and plumbers (!) and Yahoo thinks it is about Bedlam mental hospital. Perhaps that is a comment on Apple documentation.

I see similar examples on the internet that claim such code works (for example https://developer.apple.com/forums/thread/112910), but in the weeks I have been coding on iOS I have found that the majority of code examples for iOS do not compile cleanly or work, since Apple deprecates all its APIs and languages on a yearly basis (in that example from 2 years ago, Data(bytes:) is now deprecated). Fortunately Xcode often gives suggestions as to how the functions were renamed.

Surely I must be missing some tiny initialisation step that will make this work? The point cloud shader should surely support 8 bit colour channels?

Thank you very much for your kind help.

import SceneKit
...
let szChunk = 8 // Number of points: More in real-life
let arrayRGB = Array(repeating:UInt8(255), count: szChunk * 4)
let dataRGB = Data(arrayRGB)
let colorSource = SCNGeometrySource(
    data: dataRGB,
    semantic: .color,
    vectorCount: szChunk,
    usesFloatComponents: false,
    componentsPerVector: 4,
    bytesPerComponent: 1,
    dataOffset: 0,
    dataStride: 4)
let indicesArray : [Int32] = (0..<szChunk).map{Int32($0)}
let pointCloudElement = SCNGeometryElement(
    indices: indicesArray,
    primitiveType: .point)
let arrayXYZ = Array(repeating:Float(0.0), count: szChunk * 3)
let dataXYZ = arrayXYZ.withUnsafeBufferPointer { Data(buffer: $0)}
let vertexSource = SCNGeometrySource(
    data: dataXYZ,
    semantic: .vertex,
    vectorCount: szChunk,
    usesFloatComponents: true,
    componentsPerVector: 3,
    bytesPerComponent: 4,
    dataOffset: 0,
    dataStride: 12)
// This causes [SceneKit] Error: unsupported conversion uchar4 -> float4
let _ = SCNGeometry(
    sources: [vertexSource, colorSource],
    elements: [pointCloudElement])

Alternative

I note that I get the same error message "[SceneKit] Error: unsupported conversion uchar4 -> float4" if I try to use metal buffers with:

let device = MTLCreateSystemDefaultDevice()
let mtlRGB = device!.makeBuffer(bytes:arrayRGB, length:szChunk * 4)
let colorSource = SCNGeometrySource(
    buffer:mtlRGB!,
    vertexFormat: .uchar4,
    semantic: .color,
    vertexCount: szChunk,
    dataOffset: 0,
    dataStride: 4)

Workaround

However, despite not working with .uchar4 it does work with .uchar4Normalized and .uchar4Normalized_bgra ! Now it would be great if we could work out how to get it to work without using the MTLDevice (which I now get from sceneView.device) but it seems that using the above code with vertexFormat: .uchar4Normalized does work on my hardware.

1 Answers

SceneKit is based on metal framework, which means it uses texture pixel formats that metal framework does. For available texture formats that metal supports please see this link.

Related