I'm currently migrating over from Google MLKit to the Vision framework to detect and track faces in real time. Following is a simplified version of how I'm implementing the framework, but I'm also using VNDetectFaceLandmarksRequest for the landmarks and VNDetectFaceRectanglesRequest for the face angles.
private var lastObservation: VNDetectedObjectObservation?
private var sequenceRequestHandler = VNSequenceRequestHandler()
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
guard
let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer),
let lastObservation = self.lastObservation
else { return }
let request = VNTrackObjectRequest(detectedObjectObservation: lastObservation, completionHandler: handleUpdate)
do {
try sequenceRequestHandler.perform([request], on: pixelBuffer)
} catch {
print(error)
}
}
private func handleUpdate(_ request: VNRequest, error: Error?) {
DispatchQueue.main.async {
guard let newObservation = request.results?.first as? VNDetectedObjectObservation else { return }
self.lastObservation = newObservation
/// do things with the observation
}
}
My previous infrastructure assigns each detected face with a unique ID, which I want to be able to do with Vision. For example, if two people are visible in the camera, the first person's face would be assigned with 0 and the second face would be assigned with 1. This unique ID is used as the key to a dictionary so that I can store certain attributes about each face in the value. However, I can't seem to be have a property for a unique ID for each face using Vision. I have found a hashValue property of VNDetectedObjectObservation, but I'm not sure if it could be used as the unique ID for each face.