Get the cvPixelBuffer used in a VNImageRequestHandler on the VNDetectTextRectanglesRequest completion handler

user_1068522 0
Viewed 2495

I am creating my request with the following code:

let textRequest = VNDetectTextRectanglesRequest(completionHandler: 
self.detectTextHandler)
textRequest.reportCharacterBoxes = true
self.requests = [textRequest]

And inside my AVCaptureVideoDataOutputSampleBufferDelegate I am creating a VNImageRequestHandler and performing it:

let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: CGImagePropertyOrientation(rawValue: 6)!, options: requestOptions)

do {
    try imageRequestHandler.perform(self.requests)
} catch {
    print(error)
}

This gives me the results of the detection inside the handler that has the following signature:

func detectTextHandler(request: VNRequest, error: Error?)

My question is, how can i get the "cvPixelBuffer" that this request used for further processing? Am I supposed to store a temporal version of it?

apple-vision swift3 swift ios
3 Answers

https://github.com/maxvol/RxVision makes it easy without recreating the request for every image (as in the accepted answer).

let mlRequest: RxVNCoreMLRequest<CGImage> = VNCoreMLRequest.rx.request(model: model, imageCropAndScaleOption: .scaleFit)

mlRequest
    .observable
    .subscribe { [unowned self] (event) in
        switch event {
            case .next(let completion):       
                let cgImage = completion.value // NB you can easily pass the value along to the completion handler 
                if let result = completion.request.results?[0] as? VNClassificationObservation {
                    os_log("results: %@", type: .debug, result.identifier)
                }
            default:
                break
        }
    }
    .disposed(by: disposeBag)

let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, orientation: .up, options: requestOptions)
do {
    try imageRequestHandler.rx.perform([mlRequest], with: cgImage) // NB you can easily pass the value along to the completion handler
} catch {
    print(error)
}
user_8565880
Related