When using os.spawnl() or subprocess.run() functions from Python, Unicode arguments (for example, path values) gets passed as gibberish, and the called executable can't use them.
>>> subprocess.run(["ping", "ğü"], stdout=subprocess.PIPE).stdout
b'Ping request could not find host \xa7\x81. Please check the name and try again.\r\n'
PS C:\Users\flc> C:\Windows\system32\PING.EXE ğü
Ping request could not find host ğü. Please check the name and try again.
Strangely, this doesn't happen with nslookup:
>>> subprocess.run(["nslookup", "ğü"], stdout=subprocess.PIPE).stdout`
`*** one.one.one.one can't find ğü: Non-existent domain`
`b'Server: one.one.one.one\r\nAddress: 1.1.1.1\r\n\r\n'
I tested this with dir and echo, but since they require shell=True, I wasn't sure if it was making things seem OK. Hence I picked ping and nslookup.
Characters "ğü" are interpreted as "\xa7\x81" by called programs. While on Powershell, ping works with Unicode characters. Replacement characters use cp1254. It seems like even if I instruct Python to use utf-8, it passes arguments in my local encoding.
What's happening here? How can I safely pass arguments to spawned executables? Is this a problem with Python, or the target executable?