Robot.mouseMove does not work at all in Mac OS X

user_10591345 0
Viewed 4331

I created java file in IntelliJ IDE, with that code

import java.awt.AWTException;
import java.awt.MouseInfo;
import java.awt.Robot;

public class JavaRobotExample {
    public static void main(String[] args) {
        int x = 12,
                y = 300;
        try {
            Robot robot = new Robot();
            robot.mouseMove(x, y);

            int xAct = (int)    MouseInfo.getPointerInfo().getLocation().getX(),
                    yAct = (int) MouseInfo.getPointerInfo().getLocation().getY();

            String sPred = String.format("Predicted mouse location : %, d, %, d", x, y),
                    sAct = String.format("Actual mouse location : %, d, %, d", xAct, yAct);

            System.out.println(sPred);
            System.out.println(sAct);
        } catch (AWTException e) {
            e.printStackTrace();
        }
    }
}

That example works fine in Windows 7 environment, but the same code on Mac OS 10.14 doesn't move mouse using method mouseMove (but reads position with MouseInfo class).

Also I don't receive any Exception e.g.

Did someone had earlier similar problem? Any ideas how to get that code to work in Mac OS?

Best Regards,

macos-mojave mousemove awtrobot macos java
2 Answers

I'm having the same exact issue too. I have a statement "robot.mouseMove(100, 100);" and the mouse doesn't move at all. I have an idea to research. I noticed a warning or informational type of message from MacOS while I was running my code. It was asking me something about giving permission for my program to control the screen or system or something. I tried to click the "allow" checkbox, but it wouldn't let me. I'll research this some more. I think it's a MacOS thing. You (and I) have to tell MacOS that it's ok for our Java program to control the mouse.

I think this is the solution . . .

In System Preferences (the "gear" icon), under Security & Privacy, click the Privacy tab toward the top, then choose Accessibility on the left. This lists all the programs that can "control your computer". I'm using STS, not IntelliJ. I see STS listed along with BetterSnapTool and KeyCastr. Both BetterSnapTool and KeyCastr are checked. STS is not checked. And, all of these are grayed out so that I can't change any of the checks. There's a Lock icon at the bottom left of the window. I clicked on the icon, and a prompt came up asking for my password (I have some level of sysadmin privileges). I put in my password, and I am now allowed to "check" STS. I "checked" STS (you will "check" IntelliJ). After "checking" STS, I clicked the Lock icon at the bottom left again. This "closed" the lock. Now when I run my program, the robot command moves my mouse.

user_11399250

The answer is "you can" and "you can't". Let's start with the "can't".

Can't:

It seems macOS incorrectly detects the attempt to move the mouse as coming from IntelliJ, when in fact, it's coming from java. You can give IntelliJ all the permissions you want, but java will never be able to move the cursor when the permissions are given to IntelliJ.

IntelliJ (which is a Java application) is running a child process to start your program. The child process is the java command line and not the same java as is bundled with IntelliJ. This child process needs the permissions.

Can:

This is a bit round about.

  1. Use Gradle to build your project, IntelliJ recognizes Gradle projects and interacts nicely with them
  2. Remove all occurrences of IntelliJ and Java from Accessibility permissions in macOS system preferences -> Security and Privacy.
  3. Be sure all Gradle daemons are stopped (or if you haven't run Gradle yet, then be sure the next step is the first time gradle runs since you last rebooted your computer)
  4. Run you application from the command line using Gradle
  5. macOS will prompt to give permissions, open system preferences and give java permission.
  6. Run IntelliJ and run your app as a gradle project through IntelliJ, and voila!

What happens: IntelliJ, in step 6, uses the already running gradle daemon process to execute your application. This process is using java and java has permission to move the mouse cursor in macOS system preferences.

It's not pretty, but it works. If you're like me, you use gradle anyway for java projects in IntelliJ, and all you have to do is remember to run your project from the command line before running it from IntelliJ. This way, the gradle daemon (a java process) will be responsible for running the application and proper permissions will be detected by macOS.

I tested this with the following code (note a few modifications, as the OP code had some "bugs"):

import javax.swing.SwingUtilities;
import java.awt.AWTException;
import java.awt.MouseInfo;
import java.awt.Robot;
import java.lang.reflect.InvocationTargetException;

public class RunRobot {
    public static void main(String[] args) {
        int x = 12,
            y = 300;
        final int[] xAct = new int[1],
                    yAct = new int[1];
        try {
            Robot robot = new Robot();
            robot.mouseMove(x, y);
            robot.waitForIdle();

            SwingUtilities.invokeAndWait(()->{
                xAct[0] = (int) MouseInfo.getPointerInfo().getLocation().getX();
                yAct[0] = (int) MouseInfo.getPointerInfo().getLocation().getY();
            });

            String sPred = String.format("Predicted mouse location : %, d, %, d", x, y),
                   sAct  = String.format("Actual    mouse location : %, d, %, d", xAct[0], yAct[0]);

            System.out.println(sPred);
            System.out.println(sAct);
        } catch (InterruptedException | AWTException | InvocationTargetException e) {
            e.printStackTrace();
        }
    }
}
user_3618550
Related