How to dispatch multi touch gestures using AccessibilityService (disptachGesture)

user_15494398 0
Viewed 118

As we know that Android O.S. supports **multi finger gestures **. I want to develop an app that dispatches complex gesture for user. I am able to capture the motion events and dispatch gestures which are made only of one finger.

But if the user uses multiple pointers (fingers) for making a gesture, then I am able to capture them but then how can I dispatch them using Accessibility Service (dispatchGesture) function.

Any help would be most welcomed. Thanks

accessibilityservice gesture android
2 Answers

The order in which individual pointers appear within a motion event is undefined. Thus the index of a pointer can change from one event to the next, but the pointer ID of a pointer is guaranteed to remain constant as long as the pointer remains active. Use the getPointerId() method to obtain a pointer's ID to track the pointer across all subsequent motion events in a gesture. Then for successive motion events, use the findPointerIndex() method to obtain the pointer index for a given pointer ID in that motion event. For example:

private var mActivePointerId: Int = 0

override fun onTouchEvent(event: MotionEvent): Boolean {
    ...
    // Get the pointer ID
    mActivePointerId = event.getPointerId(0)

    // ... Many touch events later...

    // Use the pointer ID to find the index of the active pointer
    // and fetch its position
    val (x: Float, y: Float) = event.findPointerIndex(mActivePointerId).let { pointerIndex ->
        // Get the pointer's current position
        event.getX(pointerIndex) to event.getY(pointerIndex)
    }
    ...
}

To support multiple touch pointers, you can cache all active pointers with their IDs at their individual ACTION_POINTER_DOWN and ACTION_DOWN event time; remove the pointers from your cache at their ACTION_POINTER_UP and ACTION_UPevents. Those cached IDs may be necessary in order to handle other action events correctly; for example, when processing ACTION_MOVE event, you can find the index for each cached active pointer ID, retrieve the pointer's coordinates by using the relavent functions (getX(), getY(), etc.), then compare with your cached coordinates to discover the actually moved pointers. There can be multiple moved pointers in one ACTION_MOVE event. The getActionIndex() function does not apply to the ACTION_MOVE event.

user_55959260

So, actually to dispatch multi finger gestures using accessibility services we can use strokes for each fingers. For ex- to dispatch a two finger gesture it would be required to add two gesture stroke to gesture description and then dispatch it.

Double swipe up gesture as an example

Point position=new Point(100,10);
GestureDescription.Builder builder = new GestureDescription.Builder();
Path p = new Path();
Path q = new Path();
//For first finger
p.moveTo(position.x, position.y);
p.lineTo(position.x, position.y+300); 

//For second finger
q.moveTo(position.x, position.y);
q.lineTo(position.x + 50, position.y+300); 

//Two strokes for two fingers
builder.addStroke(new GestureDescription.StrokeDescription(p, 100L, 50L));
builder.addStroke(new GestureDescription.StrokeDescription(q, 100L, 50L));
GestureDescription gesture = builder.build();
boolean isDispatched = dispatchGesture(gesture,gestureResultCallback,null);
user_154943980
Related