I have put together an app that detects the coordinates of faces and puts a square around them using expo-camera. What I actually need to do is crop all the faces from the camera stream with 1-second interval, and pass them through a TensorFlowjs model with TensorFlow.js For React Native. I couldn't see anything like that in expo-camera documentation, is that possible with some sort of a trick?
1 Answers
I'm looking into doing something similar and I think these might help.
RNCamera a component with a onModelProcessed prop that can be used to work with the camera's stream. https://www.npmjs.com/package/react-native-tflite-camera
Here's an example I found that might help:
https://medium.com/@namar/high-performance-image-classification-with-react-native-336db0a96cd
OR
cameraWithTensors a component designed by the TF people has "the ability to yield tensors representing the camera stream." Once you have the tensors you can process them or use them as input in a tfjs model.
Here's an example using cameraWithTensors:
It sounds like you might need two model, one that detects the faces and returns the coordinates/pixel to crop at. Then after cropping, you'd need to put the cropped images into another model.
Related