I am trying to simply call a remote machine to run a script and I cannot figure out how to do this in Airflow 1.10. I do not want to have to use a key file and have put the key in the Extra section of the CONNECTION
{"private_key": "-----BEGIN RSA PRIVATE KEY-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
-----END RSA PRIVATE KEY-----"}
and here is the code from the DAG:
sshHook = SSHHook(ssh_conn_id="ssh_conn")
ls = SSHOperator(
task_id="ls",
command= "ls -l",
ssh_hook = sshHook,
dag = dag)
ls
The exact ERROR log is this:
[2021-08-16 20:20:40,704] {{logging_mixin.py:112}} INFO - [2021-08-16 20:20:40,704] {{transport.py:1819}} INFO - Connected (version 2.0, client OpenSSH_7.4)
[2021-08-16 20:20:40,753] {{taskinstance.py:1145}} ERROR - SSH operator error: No authentication methods available
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/ssh_operator.py", line 108, in execute
with self.ssh_hook.get_conn() as ssh_client:
File "/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/ssh_hook.py", line 194, in get_conn
client.connect(**connect_kwargs)
File "/usr/local/lib/python3.6/site-packages/paramiko/client.py", line 446, in connect
passphrase,
File "/usr/local/lib/python3.6/site-packages/paramiko/client.py", line 765, in _auth
raise SSHException("No authentication methods available")
paramiko.ssh_exception.SSHException: No authentication methods available
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 983, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/ssh_operator.py", line 177, in execute
raise AirflowException("SSH operator error: {0}".format(str(e)))
airflow.exceptions.AirflowException: SSH operator error: No authentication methods available
[2021-08-16 20:20:40,754] {{taskinstance.py:1202}} INFO - Marking task as FAILED.dag_id=ntam_mdl2_sync, task_id=ls, execution_date=20210816T201931, start_date=20210816T202040, end_date=20210816T202040
[2021-08-16 20:20:50,509] {{logging_mixin.py:112}} INFO - [2021-08-16 20:20:50,509] {{local_task_job.py:103}} INFO - Task exited with return code 1
Any ideas where to begin looking? This is also in AWS and the remote machine is also in the same zone.
Thanks. Justin