FocusRequestor is eating the back action forcing requiring an extra back action

I'm trying to re-create a SearchView in compose, and I'm running into an issue with focus.

I added a back handler to close the search view on back press.

When you click the search icon, it's opening up my search view and taking focus so you can begin typing, but then it takes 3 back presses to close the view instead of 2

1: Close keyboard (expected)

2: Clears focus (unexpected)

3: Triggers my back handler and closes view (expected)

The focus clearing itself is visibly eating up the back listener as it loses focus but I can't see how to clear the stack so it doesn't require an extra press? It won't go into the listener until it's eaten the 1 press.

In this below example it takes 2 instead of 1 as the keyboard isn't auto-popping up. It removes focus first, and then stops showing the view.

@Composable
fun test() {
    val isShowing = remember { mutableStateOf(true) }

    if (isShowing.value) {

        val focusRequester = FocusRequester()

        TextField(
            value = "Hello World",
            onValueChange = {},
            modifier = Modifier.focusRequester(focusRequester),
            textStyle = TextStyle(Color.White),
        )

        DisposableEffect(Unit) {
            focusRequester.requestFocus()

            onDispose {}
        }

        TestBackHandler(true) {
            isShowing.value = false
        }
    }
}

@Composable
public fun TestBackHandler(enabled: Boolean = true, onBack: () -> Unit) {
    val currentOnBack by rememberUpdatedState(onBack)

    val backCallback = remember {
        object : OnBackPressedCallback(enabled) {
            override fun handleOnBackPressed() {
                currentOnBack()
            }
        }
    }

    SideEffect {
        backCallback.isEnabled = enabled
    }

    val backDispatcher = checkNotNull(LocalOnBackPressedDispatcherOwner.current) {
        "No OnBackPressedDispatcherOwner was provided via LocalOnBackPressedDispatcherOwner"
    }.onBackPressedDispatcher

    val lifecycleOwner = LocalLifecycleOwner.current

    DisposableEffect(lifecycleOwner, backDispatcher) {
        backDispatcher.addCallback(lifecycleOwner, backCallback)

        onDispose {
            backCallback.remove()
        }
    }
}

How do i stop this extra press from being required which isn't a thing in the non compose world?