How to wrap 3d object around detected object in live tracking

Viewed 366

i have created ML Model for foot as VNRecognizedObjectObservation now i am successfully able to detect foot in live tracking the problem is i am not able to wrap or place 3d object on foot as i need 3 coordinates to place AR content.

i used below code to get bounding box after my foot detected by vision framework

func drawVisionRequestResults(_ results: [Any]) {
        CATransaction.begin()
        CATransaction.setValue(kCFBooleanTrue, forKey: kCATransactionDisableActions)
        detectionOverlay.sublayers = nil // remove all the old recognized objects

        let obs = results.first
        let final = obs

        for observation in results where observation is VNRecognizedObjectObservation {
            guard let objectObservation = observation as? VNRecognizedObjectObservation else {
                continue
            }
            // Select only the label with the highest confidence.
            let topLabelObservation = objectObservation.labels[0]
            let objectBounds = VNImageRectForNormalizedRect(objectObservation.boundingBox, Int(bufferSize.width), Int(bufferSize.height))

            let shapeLayer = self.createRoundedRectLayerWithBounds(objectBounds)

            let textLayer = self.createTextSubLayerInBounds(objectBounds,
                                                            identifier: topLabelObservation.identifier,
                                                            confidence: topLabelObservation.confidence)
            shapeLayer.addSublayer(textLayer)
            detectionOverlay.addSublayer(shapeLayer)
            //   addModel(objectBounds)
            translateCoordinate(objectBounds)
        }
        self.updateLayerGeometry()
        CATransaction.commit()
    }

but i am getting only 2d coordinates in objectBounds variable in terms of x , y height and width... i need to know 3d coordinate in ARScene space to place my 3d object ... also hit test is returning no coordinates when i input same x and y i think there is some relation in Arkit world coordinates and normal phone world coordinates

thanks in advance

0 Answers
Related