Remote control / inspect existing GTK application

user_324969 0
Viewed 121

How can I non-interactively access the widget tree of an existing GTK application? I'd like to query the coordinates of a widget, and if possible send events (hover, click, key press, …).

Using GTKInspector, I can do browse the widget tree interactively and see the size and coordinates of a widget. I'd like to access this info using a command-line tool, or using some GTK API in any language.

Screenshot of mate-calc with GTKInspector in the background.

# Steps to do this interactively, I'd like a non-interactive version.
sudo apt install libgtk-3-dev
gsettings set org.gtk.Settings.Debug enable-inspector-keybinding true
GTK_DEBUG=interactive mate-calc

# Then go to `MathWindow → GtkBox → GtkMenuBar`
# click on the (i) icon to show the details view
# `allocation` indicates the `width×height+x+y`, here it's `312×25+0+0`

Solutions I've considered:

  • Use xdotool to control the GTKInspector automatically (expect poor performance + the need to have an extra GTKInspector window on the side).
  • Use dbus, but d-feet does not show the widget tree and I suspect the things it shows about interface are about OOP or the d-bus interface, not about some user interface.
  • Back in the good old days, we could do similar things with dcop for QT applications, but of course it won't work for GTK.
gtk3 gtk2 ui-automation gtk widget
1 Answers

dogtail uses assistive technologies to access an application's widget tree. Tested on Xubuntu 20.04.

$ sudo apt install python3-dogtail
$ sniff # explore using a GUI, it will enable assistive technologies for you.

dogtail-example.py:

#!/usr/bin/env python3
from dogtail.tree import *
app = root.application('mate-calc')
menuBar = app.findChildren(predicate.GenericPredicate(roleName="menu bar"))[0]
print('On-screen coordinates: ', menuBar.position)
print('Size:                  ', menuBar.size)
# to preview the nodes available in the menu bar, use menuBar.dump()

Output:

$ python3 dogtail-example.py
On-screen coordinates: (529, 431)
Size:                  (312, 25)

There are a few examples in /usr/share/doc/python3-dogtail/examples/ but the actual "doc" is in the docstrings of /usr/lib/python3/dist-packages/dogtail/tree.py and other files in that directory.

user_3249690
Related