Android AccessibilityService performAction() method not working

user_4882274 0
Viewed 218

I am developing an accessibility service for Android. The service calls an app, and that app has a RecyclerView. Then I want to click on an element of the RecyclerView with performAction(AccessibilityNodeInfo.ACTION_CLICK) but it is not working. I know there are a few similar questions but none of them works for me. Also I checked the official documentation for the class of the performAction method https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo

This is my code:

@Override
public void onAccessibilityEvent(Accessibility event){
    AccessibilityNodeInfo source = event.getSource();
    if(source != null){
        List<AccessibilityNodeInfo> list = source.findAccessibilityNOdeInfosByText("mystring");

        list.get(0).performAction(AccessibilityNodeInfo.ACTION_CLICK);
    }
}

This is my configuration xml file:

<accessibility-srvice xmlns...
    android:accessibilityFeedbackType = "feedbackGeneric"
    android:AccessibilityFlags = "flagDefault"
    android:canPerformGestures = "true"
    android:canRetrieveWIndowCOntent = "true"

I think I misunderstood something, but i don't know what can be. Any help is appreciated.

accessibilityservice accessibility android
1 Answers

The simple answer is that while finding the node by text is fine, that particular node was not the node with the desired onClick event. The solution is to call

list.get(0).getParent().performAction(AccessibilityNodeInfo.ACTION_CLICK)

The clarifying discussion is below

I think .performAction(AccessibilityNodeInfo.ACTION_CLICK) is right, but there might be some other concerns. Sorry for posting as an answer but a comment is too small.

  1. Are you sure the onAccessibilityEvent is being called? I don't think that is the right event, but I can't be sure. Maybe put a log in there to ensure it's calling the event when you expect it to be called.

  2. Also, looking at the source might restrict your search, maybe instead of event.getSource() try using rootInActiveWindow (I use Kotlin so it might have a method, see https://developer.android.com/reference/android/accessibilityservice/AccessibilityService#getRootInActiveWindow(int))

EDIT: 28 March 2022

I have run this code on my own accessibility service and it does click the button. But it's very prone to overflow.

    var ranOnce = false // prevent overflow
    override fun onAccessibilityEvent(event: AccessibilityEvent?) {
        if (event == null) return

        if (event.eventType == TYPE_WINDOW_STATE_CHANGED) return

        if (event.source != null && !ranOnce) {
            val nodeList = rootInActiveWindow.findAccessibilityNodeInfosByText("Menu")
            //event.source.findAccessibilityNodeInfosByText("Menu") // <-- always nothing in list

            Log.d("onAccessibilityEvent", "List of nodes: $nodeList")
            if (nodeList.size > 0) {
                android.util.Log.d("onAccessibilityEvent", "Node info: ${nodeList[0]}")
                ranOnce = true
                nodeList[0].performAction(AccessibilityNodeInfo.ACTION_CLICK) //<-- caused an infinite loop!
            } else {
                Log.d("onAccessibilityEvent", "No nodes found")
            }
        } else {
            Log.d("onAccessibilityEvent", "event.source is null!")
        }
    }
user_9320520
Related