How to remove keyboard flicker when switching between TextFields?

I have a LazyColumn consisting of text fields with numeric input. When I switch between text fields, I sometimes see a flicker. It first opens the normal text keyboard and then switches to numeric one. This is leading to a really bad UX.

My code:

LazyColumn {
    items(payers) {
        Row {
            Image(...)
            Text(...)
            Box(
                contentAlignment = Alignment.Center,
                modifier = Modifier
                    .border(1.dp, Color.Gray, RoundedCornerShape(4.dp))
                    .padding(vertical = 4.dp)
            ) {
                BasicTextField(
                    value = it.amount,
                    onValueChange = { /* Update it.amount */ },
                    keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
                )
                if (it.amount.isEmpty())
                    Text(
                        text = "₹ 0",
                        modifier = Modifier.alpha(0.5f)
                    )
            }
        }
    }
}

Result: