How to detect what image part was clicked in Android Compose

Question

I am trying to get what part of the image has been clicked and get the Color of that point

I'm able to get the tap coordinates through a Modifier but I don't know how to relate it with the drawable of the Image:

@Composable
fun InteractableImage(
   modifier = Modifier
   ...
) {
  val sectionClicked by rememberSaveable { mutableStateOf(Offset.Unspecified) }
  
  Image(
     modifier = Modifier.fillMaxSize().pointerInput(Unit) {
        detectTapGestures (
           onTap = { offset -> sectionClicked = offset },
     painter = painterResource(id = R.drawable.ic_square),
     contentDescription = "square"
  )
  
  ....
}

With the classic view system, I could access the matrix of the imageView, and with the scale property and the intrinsic drawable dimensions, I could find the part of the image that was clicked. How could I achieve this using Android Compose?

user_5457853 · Answer

It's a little bit long but working solution except when there are spaces at edges of the Bitmap inside Image because of contentScale param of Image. Need to do another operation for calculating space around but couldn't find it at the moment. And if anyone finds the space just set it to startX, startY and use linear interpolation.

1- Instead of painter get an ImageBitmap

val imageBitmap: ImageBitmap = ImageBitmap.imageResource(
    LocalContext.current.resources,
    R.drawable.landscape7
)

val bitmapWidth = imageBitmap.width
val bitmapHeight = imageBitmap.height

2- Create variables for touch position on x, y axes and size of the Image that we put bitmap into

var offsetX by remember { mutableStateOf(0f) }
var offsetY by remember { mutableStateOf(0f) }
var imageSize by remember { mutableStateOf(Size.Zero) }

Get Image width from Modifier.onSizeChanged{imageSize = it.toSize()}

3- Get touch position scale it to bitmap position, convert to pixel and return r, g, b values from pixel

.pointerInput(Unit) {
    detectTapGestures { offset: Offset ->
        // Touch coordinates on image
        offsetX = offset.x
        offsetY = offset.y

        // Scale from Image touch coordinates to range in Bitmap
        val scaledX = (bitmapWidth/imageSize.width)*offsetX
        val scaledY = (bitmapHeight/imageSize.height)*offsetY

            try {
                val pixel: Int =
                    imageBitmap
                        .asAndroidBitmap()
                        .getPixel(scaledX.toInt(), scaledY.toInt())


                // Don't know if there is a Compose counterpart for this
                val red = android.graphics.Color.red(pixel)
                val green = android.graphics.Color.green(pixel)
                val blue = android.graphics.Color.blue(pixel)

                colorInTouchPosition = Color(red,green,blue)
            }catch (e:Exception){
                println("Exception e: ${e.message}")
            }
    }
}

4- Instead of scaling and lerping you can just use val scaledX = (bitmapWidth/endImageX)*offsetX but when your bitmap is not fit into image you need to use linear interpolation after calculating space on left, right, top or bottom of the image, or you should make sure that Image has the same width/height ratio as Bitmap.

i used contentScale = ContentScale.FillBounds for simplicity

// Scale from Image touch coordinates to range in Bitmap
val scaledX = (bitmapWidth/imageSize.width)*offsetX
val scaledY = (bitmapHeight/imageSize.height)*offsetY

Full Implementation

@Composable
private fun TouchOnImageExample() {

    val imageBitmap: ImageBitmap = ImageBitmap.imageResource(
        LocalContext.current.resources,
        R.drawable.landscape6
    )

    val bitmapWidth = imageBitmap.width
    val bitmapHeight = imageBitmap.height

    var offsetX by remember { mutableStateOf(0f) }
    var offsetY by remember { mutableStateOf(0f) }
    var imageSize by remember { mutableStateOf(Size.Zero) }

    // These are for debugging
    var text by remember { mutableStateOf("") }
    var colorInTouchPosition by remember { mutableStateOf(Color.Unspecified) }

    val imageModifier = Modifier
        .background(Color.LightGray)
        .fillMaxWidth()
        // This is for displaying different ratio, optional
        .aspectRatio(4f / 3)
        .pointerInput(Unit) {
            detectTapGestures { offset: Offset ->
                // Touch coordinates on image
                offsetX = offset.x
                offsetY = offset.y

// Scale from Image touch coordinates to range in Bitmap
val scaledX = (bitmapWidth/imageSize.width)*offsetX
val scaledY = (bitmapHeight/imageSize.height)*offsetY


                // TODO This section needed when Bitmap does not fill Image completely
                // However i couldn't find a solution to find spaces correctly

//                // Need to calculate spaces at edges of the bitmap inside Image Composable if
//                // not exactly filling the bounds of Image
//                val startImageX = 0f
//                val startImageY = 0f
//
//                // End positions, this might be less than Image dimensions if bitmap doesn't fit Image
//                val endImageX = imageSize.width - startImageX
//                val endImageY = imageSize.height - startImageY
//                val scaledX =
//                    scale(
//                        start1 = startImageX,
//                        end1 = endImageX,
//                        pos = offsetX,
//                        start2 = 0f,
//                        end2 = bitmapWidth.toFloat()
//                    ).coerceAtMost(bitmapWidth.toFloat())

//                val scaledY =
//                    scale(
//                        start1 = startImageY,
//                        end1 = endImageY,
//                        pos = offsetY,
//                        start2 = 0f,
//                        end2 = bitmapHeight.toFloat()
//                    ).coerceAtMost(bitmapHeight.toFloat())

                    try {
                        val pixel: Int =
                            imageBitmap
                                .asAndroidBitmap()
                                .getPixel(scaledX.toInt(), scaledY.toInt())


                        // Don't know if there is a Compose counterpart for this
                        val red = android.graphics.Color.red(pixel)
                        val green = android.graphics.Color.green(pixel)
                        val blue = android.graphics.Color.blue(pixel)


                        text = "Image Touch: $offsetX, offsetY: $offsetY
" +
                                "size: $imageSize
" +
                                "bitmap width: ${bitmapWidth}, height: $bitmapHeight
" +
                                "scaledX: $scaledX, scaledY: $scaledY
" +
                                "red: $red, green: $green, blue: $blue
"

                        colorInTouchPosition = Color(red,green,blue)
                    }catch (e:Exception){
                        println("Exception e: ${e.message}")
                    }
            }
        }
        .onSizeChanged { imageSize = it.toSize() }


    Image(
        bitmap = imageBitmap,
        contentDescription = null,
        modifier = imageModifier,
        contentScale = ContentScale.FillBounds
    )
    Text(text = text)
    Box(
        modifier = Modifier
            .then(
                if (colorInTouchPosition == Color.Unspecified) {
                    Modifier
                } else {
                    Modifier.background(colorInTouchPosition)
                }
            )
            .size(100.dp)
    )
}

/**
 * Interpolate position x linearly between start and end
 */
fun lerp(start: Float, end: Float, amount: Float): Float {
    return start + amount * (end - start)
}

/**
 * Scale x1 from start1..end1 range to start2..end2 range
 */
fun scale(start1: Float, end1: Float, pos: Float, start2: Float, end2: Float) =
    lerp(start2, end2, calculateFraction(start1, end1, pos))

/**
 * Calculate fraction for value between a range [end] and [start] coerced into 0f-1f range
 */
fun calculateFraction(start: Float, end: Float, pos: Float) =
    (if (end - start == 0f) 0f else (pos - start) / (end - start)).coerceIn(0f, 1f)

Result

How to detect what image part was clicked in Android Compose

1 Answers