I have set up a Remote Interpreter to a Server from PyCharm. The connection is working fine and i have also configured the Deployment from my local machine to this server.
The interpreter i configured is from Spark (/usr/bin/spark-submit). But, when i run my code, it outputs the following error:
Error: Unrecognized option: -u
Below is the command that is being sent to the remote server:
ssh://user@MYHOST:22/usr/bin/spark-submit -u /home/user/pycharm/main.py
I have tried with the Python Interpreter, using the Python from the remote server and it worked just fine. The problem is that spark-submit does not accept the option -u but i cannot remove that option from PyCharm. Also, i've removed the PYTHONUNBUFFERED=1 environment variable from the Run Configurations in PyCharm, but it did not work.
Field which i have removed the PYTHONUNBUFFERED=1 option
Parameters Field in Run Configurations:
I have also searched the JetBrains forum, but didn't find anything that refers to this problem.