How to access text in UI element with JXA?

user_3343783 0
Viewed 362

When you open a Terminal or iTerm window, you are shown a string like

Last login: Fri Jul 10 00:00:00 on ttys000
(base) my_username@my_hostname ~ %

I would like to be able to read this string into a variable in JXA.

enter image description here

This is how far I got until now.

1. Accessibility Inspector

With Accessibility Inspector I have discovered the hierarchy of UI elements which lead to the string I am looking for. enter image description here

2. UI Browser

I have used UI Browser to navigate trough the hierarchy and obtain an Apple Script code to access the element; but I need JXA code.

enter image description here

3. Translating into code

In order to try to access the above UI hierarchy with JXA I have first enabled the Safari JavaScript debugger as explained here and then I have written this code to invoke the debugger:

i_term = Application('iTerm')
i_term.quit()
i_term.activate()
delay(0.2)
debugger

Now, within the console of the debugger, I have tried to access the object by using the same variable naming shown in the first code snippet from this answer, that is

i_term.windows()[0].groups()[0].splitterGroups()[0].scrollAreas()[0]

At this point I have two problems:

  • I actually don't know if the object that I am trying accessing exists, because in the console I get uninformative output. I just know that i_term.windows()[0] is correct since if I type i_term.windows()[0].name() I get the string "zsh", which is the title of the window.
  • I don't know what's the name of the method/variable for the text UI element.

More generally, how I am describing in this other question (link coming in a edit I will soon make), I have tried printing in many ways the "content" of the variables I am working with, but I can't see the methods/values they contain. For instance: I would like to be able to see the list of methods/variables of i_term, in particular to know that it contains the method windows(), but I have found no way of getting this information. Being able to get this information would make possible to write the code much more easily, even with the current lack of documentation.

How can I obtain the string value of that text?

javascript-automation
1 Answers

Why are you wanting to achieve your goal through UI scripting ? It's the most difficult kind of script to implement well, and the most likely to break.

I don't know whether you're aware, but the contents of a terminal window in both Terminal.app and iTerm.app can be retrieved through their AppleScript properties.

iTerm:

Application('com.googlecode.iterm2').windows[0].tabs[0].sessions[0].contents()
// OR:                              .currentWindow.currentSession.contents()

Terminal:

Application('com.apple.Terminal').windows[0].tabs[0].contents()
// OR:                           .windows[0].selectedTab.contents()

Side-bar:

To obtain an object reference—if one exists—to any given UI element on your screen, you need:

  1. A definitive, unobstructed pixel belonging solely to the UI element in question, for which you can discern its (x, y) coordinates relative to the top-left of your screen. The "l" of "login" seems good:

  2. Access privileges granted for Script Editor and System Events.

Then, when you run the command:

tell application id "com.apple.systemevents" to click at {x, y}

and, provided that the pixel you identified at coordinates (x, y) remains in view when the command executes, then the return value of the command will be the object reference to the UI element occupying the pixel at those coordinates.

If it returns missing value, the element isn't accessible.

user_90510060
Related