Is there a way to crop Image/ImageProxy (before passing to MLKit's analyzer)?

Yes, it's true that if you use ViewPort and set viewport to yours UseCases(imageCapture or imageAnalysis as here https://developer.android.com/training/camerax/configuration) you can get only information about crop rectangle especially if you use ImageAnalysis(because if you use imageCapture, for on-disk the image is cropped before saving and it doesn't work for ImageAnalysis and if you use imageCapture without saving on disk) and here solution how I solved this problem:

1. First of all set view port for use cases as here: https://developer.android.com/training/camerax/configuration

2. Get cropped bitmap to analyze

   override fun analyze(imageProxy: ImageProxy) {
        val mediaImage = imageProxy.image
        if (mediaImage != null && mediaImage.format == ImageFormat.YUV_420_888) {
            croppedBitmap(mediaImage, imageProxy.cropRect).let { bitmap ->
                requestDetectInImage(InputImage.fromBitmap(bitmap, rotation))
                    .addOnCompleteListener { imageProxy.close() }
            }
        } else {
            imageProxy.close()
        }
    }
   
    private fun croppedBitmap(mediaImage: Image, cropRect: Rect): Bitmap {
        val yBuffer = mediaImage.planes[0].buffer // Y
        val vuBuffer = mediaImage.planes[2].buffer // VU
   
        val ySize = yBuffer.remaining()
        val vuSize = vuBuffer.remaining()
   
        val nv21 = ByteArray(ySize + vuSize)
   
        yBuffer.get(nv21, 0, ySize)
        vuBuffer.get(nv21, ySize, vuSize)
   
        val yuvImage = YuvImage(nv21, ImageFormat.NV21, mediaImage.width, mediaImage.height, null)
        val outputStream = ByteArrayOutputStream()
        yuvImage.compressToJpeg(cropRect, 100, outputStream)
        val imageBytes = outputStream.toByteArray()
   
        return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
    }
   

Possibly there is a loss in conversion speed, but on my devices I did not notice the difference. I set 100 quality in method compressToJpeg, but mb if set less quality it can improve speed, it need test.

upd: May 02 '21 :

I found another way without convert to jpeg and then to bitmap. This should be a faster way.

1. Set viewport as previous.

2. Convert YUV_420_888 to NV21, then crop and analyze.

    override fun analyze(imageProxy: ImageProxy) {
        val mediaImage = imageProxy.image
        if (mediaImage != null && mediaImage.format == ImageFormat.YUV_420_888) {
            croppedNV21(mediaImage, imageProxy.cropRect).let { byteArray ->
                requestDetectInImage(
                    InputImage.fromByteArray(
                        byteArray,
                        imageProxy.cropRect.width(),
                        imageProxy.cropRect.height(),
                        rotation,
                        IMAGE_FORMAT_NV21,
                    )
                )
                    .addOnCompleteListener { imageProxy.close() }
            }
        } else {
            imageProxy.close()
        }
    }
   
    private fun croppedNV21(mediaImage: Image, cropRect: Rect): ByteArray {
        val yBuffer = mediaImage.planes[0].buffer // Y
        val vuBuffer = mediaImage.planes[2].buffer // VU
   
        val ySize = yBuffer.remaining()
        val vuSize = vuBuffer.remaining()
   
        val nv21 = ByteArray(ySize + vuSize)
   
        yBuffer.get(nv21, 0, ySize)
        vuBuffer.get(nv21, ySize, vuSize)
   
        return cropByteArray(nv21, mediaImage.width, cropRect)
    }
   
    private fun cropByteArray(array: ByteArray, imageWidth: Int, cropRect: Rect): ByteArray {
        val croppedArray = ByteArray(cropRect.width() * cropRect.height())
        var i = 0
        array.forEachIndexed { index, byte ->
            val x = index % imageWidth
            val y = index / imageWidth
   
            if (cropRect.left <= x && x < cropRect.right && cropRect.top <= y && y < cropRect.bottom) {
                croppedArray[i] = byte
                i++
            }
        }
   
        return croppedArray
    }
   

First crop fun I took from here: Android: How to crop images using CameraX?

And I found also another crop fun, it seems that it is more complicated:

private fun cropByteArray(src: ByteArray, width: Int, height: Int, cropRect: Rect, ): ByteArray {
    val x = cropRect.left * 2 / 2
    val y = cropRect.top * 2 / 2
    val w = cropRect.width() * 2 / 2
    val h = cropRect.height() * 2 / 2
    val yUnit = w * h
    val uv = yUnit / 2
    val nData = ByteArray(yUnit + uv)
    val uvIndexDst = w * h - y / 2 * w
    val uvIndexSrc = width * height + x
    var srcPos0 = y * width
    var destPos0 = 0
    var uvSrcPos0 = uvIndexSrc
    var uvDestPos0 = uvIndexDst
    for (i in y until y + h) {
        System.arraycopy(src, srcPos0 + x, nData, destPos0, w) //y memory block copy
        srcPos0 += width
        destPos0 += w
        if (i and 1 == 0) {
            System.arraycopy(src, uvSrcPos0, nData, uvDestPos0, w) //uv memory block copy
            uvSrcPos0 += width
            uvDestPos0 += w
        }
    }
    return nData
}

Second crop fun I took from here: https://www.programmersought.com/article/75461140907/

I would be glad if someone can help improve the code.

I'm still improving the way to do it. But this will work for me now

CameraX crop image before sending to analyze

<androidx.constraintlayout.widget.ConstraintLayout
  android:layout_width="match_parent"
  android:layout_height="wrap_content"
  android:paddingBottom="@dimen/_40sdp">

  <androidx.camera.view.PreviewView
      android:id="@+id/previewView"
      android:layout_width="match_parent"
      android:layout_height="0dp"
      app:layout_constraintDimensionRatio="1:1"
      app:layout_constraintEnd_toEndOf="parent"
      app:layout_constraintStart_toStartOf="parent"
      app:layout_constraintTop_toTopOf="parent" /></androidx.constraintlayout.widget.ConstraintLayout>

Cropping an image into 1:1 before passing it to analyze

                override fun onCaptureSuccess(image: ImageProxy) {
                    super.onCaptureSuccess(image)

                    var bitmap: Bitmap = imageProxyToBitmap(image)
                    val dimension: Int = min(bitmap.width, bitmap.height)
                    bitmap = ThumbnailUtils.extractThumbnail(bitmap, dimension, dimension)

                    imageView.setImageBitmap(bitmap) //Here you can pass the crop[from the center] image to analyze

                    image.close()

                }

**Function for converting into bitmap **

    private fun imageProxyToBitmap(image: ImageProxy): Bitmap {
    val buffer: ByteBuffer = image.planes[0].buffer
    val bytes = ByteArray(buffer.remaining())
    buffer.get(bytes)
    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

You would use ImageProxy.SetCroprect to get the rect and then use CropRect to set it. For example if you had imageProxy, you would do : ImageProxy.setCropRect(Rect) and then you would do ImageProxy.CropRect.