Mouse movements for the entire computer can be done using SendInput(MOUSEEVENTF_MOVE), which can be manipulated when the UE4 window is activated.
Given that there are multiple UE4 Windows, it may be necessary to have a transiting service to handle them uniformly, but this may require switching applications or even desktops.
It would be nice if I could send mouse movement information to the specified UE4 process. Unfortunately, keyboard input and mouse clicks can be done with WM_KEYDOWN/WM_LBUTTONDOWN, etc., but mouse movement alone cannot be done with WM_MOUSEMOVE
I read the source code of UE4 and found that it uses VM_INPUT + GetDeviceData for mouse movement events, because the definition of RAWINPUT is invisible, so I can't simulate the message of RAWINPUT. In addition, Microsoft has implemented GlobalAlloc as LocalAlloc So, even if it can be simulated, it cannot be injected into different processes. Even if it can be injected into the process, I can’t modify the usage method of GetDeviceData in the UE4 source code.
I heard that there is a technology called "pixel streaming" in UE4, we can use webrtc to directly operate remote UE4 applications, including mouse movement, I tested it, it is true, but I don’t know what message is sent in webrtc, If I know, maybe I can send it a similar message to operate it.
So, there is any way to operator multiple UE4 processes at same time without webrtc?