Translate Firebase MLKit Bounding box coordinates to screen view coordinates

user_599346 0
Viewed 2653

I am using the FirebaseVision Object detection to detect things from the CameraX camera preview. It is detecting things find but I am trying to draw the bounding box of the items detected over the camera preview. In doing so the bounding box that firebase gives back is not for the image itself not the preview view to they appear in the wrong place.

The image size that I get back from firebase is 1200x1600 and the preview size is 2425x1440

How do I translate the bounding boxes returned from firebase to the correct screen coordinates?

firebase-mlkit android-camerax android
3 Answers

What I ended up doing was I took the image size that the camera took, divided the width/height by the view width/height to get the scale size

if(isPortraitMode()){
    _scaleY = overlayView.height.toFloat() / imageWidth.toFloat()
    _scaleX = overlayView.width.toFloat() / imageHeight.toFloat()
}else{
    _scaleY = overlayView.height.toFloat() / imageHeight.toFloat()
    _scaleX = overlayView.width.toFloat() / imageWidth.toFloat()
}

Now that I have the scale I can then take the bounding box return by the firebase detector and translate the x and y coordinates by the scales

private fun translateX(x: Float): Float = x * _scaleX
private fun translateY(y: Float): Float = y * _scaleY

private fun translateRect(rect: Rect) = RectF(
    translateX(rect.left.toFloat()),
    translateY(rect.top.toFloat()),
    translateX(rect.right.toFloat()),
    translateY(rect.bottom.toFloat())
)

Which then gives you the scaled rect coordinates which you then draw on the screen

user_5993460
user_2773680

Thanks @tyczj,

Your answer help me find my solution, let me add if someone is using front camera like me for Face detection you need to inverte the x axis, example:

val previewSize = overlayView.width.toFloat()
val newLeft = if (isFrontCamera) previewSize - (rect.right * scaleX) else rect.left * scaleX
val newRight = if (isFrontCamera) previewSize - (rect.left * scaleX) else rect.right * scaleX
user_15751330
Related