Jetpack navigation compose, bottom nav, make back button switch tabs?

I'm trying to use bottom navigation (ie, tabs) with Android Jetpack Compose and the Navigation Compose library. I have it set up with multiple back stacks, so that the state of each tab is preserved when you switch between them. However, if I change tabs by tapping one of them, the Android back button does not go back to the previous tab. Instead it navigates within that tab. How do I set it up to go back to previous tab?

Example scenario (I want to change behavior of last step).

1. I have two tabs, A and B.
2. Start in the A tab, on screen A1.
3. Push a new screen in that tab, called A2.
4. Tap the B tab to move to it, on its start screen B1.
5. Push a new screen in that tab, B2. So the tab stacks are now: A:[A1,A2], and B:[B1, B2]
6. Tap the A tab. I'm back in that stack, on screen A2. The state in that tab was preserved.
7. Press Android back button. It goes to A1. Instead I want it to go back to the B tab, screen B2.

MyAppBottomNavItem.all.forEach { item ->
    BottomNavigationItem(
        icon = { Icon(item.icon, contentDescription = null) },
        label = {
            Text(item.name, maxLines = 1, textAlign = TextAlign.Center,
                 overflow = TextOverflow.Visible, softWrap = false,
                 fontSize = 10.sp)
        },
        selected = currentDestination?.hierarchy?.any { it.route == item.route } == true,
        alwaysShowLabel = true,
        onClick = {
            nav.navigate(item.route) {
                popUpTo(nav.graph.findStartDestination().id) {
                    saveState = true
                }
                launchSingleTop = true
                restoreState = true
            }
        }
    )
}