How to execute SQL in redshift with Airflow (version 1)

user_3934598 0
Viewed 964

Is there a way to trigger a stored procedure in redshift from airflow? Best scenario would be to not use a python operator, but I haven't found a reshift operator in airflow version < 2.

airflow
1 Answers

In Airflow <2.1 there is no RedShiftSqlOperator because RedShift is compatible with PostgreSQL so you can just use PostgresOperator:

from airflow.providers.postgres.operators.postgres import PostgresOperator
PostgresOperator(
        sql='SELECT * FROM my_table',
        postgres_conn_id='redshift_default',
        task_id='task',
    )

That said, in more recent versions (Airflow >= 2.1) since some users were struggling with this question as well (See GitHub issue) Airflow added RedshiftSQLOperator which is available in Amazon provider version 2.4.0 :

pip install apache-airflow-providers-amazon>=2.4.0

Then you can use the operator as:

from airflow.providers.amazon.aws.operators.redshift import RedshiftSQLOperator
    setup__task_create_table = RedshiftSQLOperator(
        task_id='task',
        redshift_conn_id="redshift_default"
        sql="SELECT * FROM my_table",
    )
user_146244090
Related