Espresso - onClick does not work when performed on a nested ButtonView inside a Compose AndroidView

I'm using Espresso to test a Compose screen that has a nested ButtonView in an AndroidView composable. However calling Espresso.onView(ViewMatchers.withText("Click me")).perform(ViewActions.click()) does not work and the onClick callback is not called when the test is run, leading to the assertion to check if text has been updated to fail.

My compose/testing dependencies are

implementation "androidx.compose.ui:ui:1.1.1"
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
androidTestImplementation "androidx.compose.ui:ui-test-junit4:1.1.1"
debugImplementation "androidx.compose.ui:ui-tooling:1.1.1"
debugImplementation "androidx.compose.ui:ui-test-manifest:1.1.1"

My hybrid Compose code looks like this

// OnClick functionality works when  running app
class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            SampleAppTheme {
                HybridSampleScreen()
            }
        }
    }
}

@Composable
fun HybridSampleScreen() {
    var text by rememberSaveable { mutableStateOf("Initial Text") }
    Surface(
        modifier = Modifier.fillMaxSize(),
        color = MaterialTheme.colors.background
    ) {
        Column() {
            ComposableWithAndroidButton {
                text = "I've been clicked"
            }
            Text(text)
        }
    }
}

@Composable
fun ComposableWithAndroidButton(onButtonClick: () -> Unit) {
    Column() {
        Text(text = "Hello World")
        AndroidView(factory = { context ->
            //This is an alias so I can use Button view and Button Composable in the same file
            ClassicViewButton(context).apply {
                setOnClickListener { onButtonClick.invoke() }
                text = "Click me"
            }
        })
    }
}

Failing Test

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {

    @get:Rule
    val composeTestRule = createComposeRule()

    @Test
    fun testAndroidViewButtonInComposable() {
        composeTestRule.setContent {
            SampleAppTheme {
                HybridSampleScreen()
            }
        }

        composeTestRule.onNodeWithText("Hello World").assertIsDisplayed()
        // The first two assertions pass so it looks like Espresso is aware that there is a clickable Button view
        Espresso.onView(ViewMatchers.withText("Click me")).check(matches(isDisplayed()))
        Espresso.onView(ViewMatchers.withText("Click me")).check(matches(isClickable()))
        Espresso.onView(ViewMatchers.withText("Click me")).perform(ViewActions.click())

        composeTestRule.waitForIdle()

        composeTestRule.onNodeWithText("I've been clicked").assertIsDisplayed()
    }
}

To sanity-check I also created a Screen that's fully using Composables and created a test for it and that passes for me.

Full Compose Screen

@Composable
fun FullComposeScreen() {
    var text by rememberSaveable { mutableStateOf("Initial Text") }
    Column {
        ComposableWithComposeButton {
            text = "I've been clicked"
        }
        Text(text = text)
    }
}

@Composable
fun ComposableWithComposeButton(onButtonClick: () -> Unit) {
    Column() {
        Text(text = "Hello World")
        Button(onClick = {
            onButtonClick.invoke()
        }) {
            Text(text = "I'm a composable click me")
        }
    }
}

Passing Test

@Test
fun testFullComposeButtonInComposable() {
    composeTestRule.setContent {
        SampleAppTheme {
            FullComposeScreen()
        }
    }

    composeTestRule.onNodeWithText("Hello World").assertIsDisplayed()

    composeTestRule.onNodeWithText("I'm a composable click me").assertIsDisplayed()
    composeTestRule.onNodeWithText("I'm a composable click me").assertHasClickAction()
    composeTestRule.onNodeWithText("I'm a composable click me").performClick()

    composeTestRule.waitForIdle()

    composeTestRule.onNodeWithText("I've been clicked").assertIsDisplayed()
}