No module named 'airflow.providers.ssh' on AWS Airflow (Amazon MWAA)

user_6767842 0
Viewed 3241

I need to use sshoperator in a DAG on AWS Airflow (Amazon MWAA), so I imported the following library in my DAG file

from airflow.contrib.operators.ssh_operator import SSHOperator

It seems sshoperator has been defined in paramiko library, so I have added the following modules to requiremets.txt file as well

  • paramiko>=2.6.0
  • pysftp>=0.2.9
  • sshtunnel>=0.1.4

But still, I'm getting the following error when I browse DAGs

Broken DAG: 
  File "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/ssh_operator.py", 
  line from airflow.providers.ssh.operators.ssh import SSHOperator
  ModuleNotFoundError: No module named 'airflow.providers.ssh'

My underestanding was sshoperator in an essential and built in operators in Airflow, so technically I shouldn't add any extra libraries or plugins to Airflow.

mwaa airflow
2 Answers

This command fixed it for me:

pip install apache-airflow-providers-ssh
user_104996240

Was researching something similar and believe that I found the answer to both of our questions... According to Docker Questions...

Providers package is no longer included with Airflow, but you can separately install them with pip using the specific backport package

It looks like there is a backport for ssh here. So probably just have to add that to your requirements.txt?

user_63823770
Related