Airflow SQLOperator/Sensor vs MsSqlOperator

user_16982984 0
Viewed 90

For apache-airflow 3.2.0

I am looking to use an operator to connect to an MsSql database. I found both SQLOperator and MsSqlOperator.

Does both work for an MsSql database or is it only the MsSqlOperator that works for this type of connection?

If both works, would there be any benefit of using the MsSqlOperator? There exists an SQLSensor, but no MsSqlSensor, would that make it wiser to use SQLOperator/Sensor instead of implementing your own MsSqlSensor (assuming SQLOperator/Sensor works for MsSql database)?

If anybody has any guidance on the differences and benefits between these two operators (and sensor) it would be appreciated.

airflow sql-server
1 Answers

You asked two separate questions so I'll tackle them separately.

MsSqlOperator vs SqlOpeartor:

There is no SqlOpeartor in the Airflow. There was an attempt to add one in PR but it was not merged(*). You should use MsSqlOperator.

(*) Currently it's not easy to create a generic SqlOperator as different databases handle multiple SQL statements in different ways I explained the challenge in this Github Issue this is currently a blocker for creating generic SqlOperator. Once the blocker is resolved Airflow will probably try again to create a generic operator.

MsSqlSensor vs SqlSensor:

There is no MsSqlSensor. You should use SqlSensor which is compatible with MsSql database. Unlike previous paragraph in sensor we run a single SQL statement. In that case it's easy to create a generic sensor to serve multiple DBs without the need to create a custom class per each provider (MsSql, MySql, PostgresSql etc..)

Noting: Should there is a need you can always extend the current operators and add/adjust the functionality you required. This is the power of writing workflows as code.

user_146244090
Related