UI Automator with custom keyboard. Find exact keyboard button and click it

user_4673764 0
Viewed 458

I'm trying to write UI automation tests for the custom soft keyboard using UI Automator and\or Espresso. Tried different ways but I can't find a proper solution to "find the exact button on the opened keyboard and click it".

Problems:

  1. UIAtomator's UiDevice.findObject(By.text("Q")).click() doesn't find Q button on keyboard.
  2. Espresso's onView(withText("Q")).perform(click()) doesn't find the button either.

For now, it looks like the only way to click button is to measure XY coordinates based on screen height and keyboard height. But this solution is ugly and not persistent.

typeText("text")and uiObject.text = "text" don't work since it bypasses keyboard input.

Was anyone working with custom keyboards? Please help.

android-uiautomator android-espresso ui-automation android
2 Answers

Since you're building a custom soft keyboard then I expect you're using a KeyboardView. KeyboardView draws the keys using a canvas therefore it is not possible to get the resource ids of the keys... so no chance to find them through the UiDevice's findObject method.

Considering the KeyboardView class is deprecated since API 29, a possible solution will be to reimplement your own KeyboardView (as suggested here) and use AccessibilityNodeInfo class to build virtual elements (one for each key) that will be included into the view hierarchy.

user_24428310
user_136023410
Related