What is a difference between op_kwargs and templates_dict od PythonOperator in Airflow?

user_11914526 0
Viewed 188

I am wonderind what is a difference in usage between op_kwargs and templates_dict in Airflow, since both are templated fields in PythonOperator template_fields= ['templates_dict', 'op_args', 'op_kwargs'].

I checked the documentation but it's still not clear for me:

op_kwargs (Optional[Mapping[str, Any]]) – a dictionary of keyword arguments that will get unpacked in your function

templates_dict (Optional[Dict[str, Any]]) – a dictionary where the values are templates that will get templated by the Airflow engine sometime between __init__ and execute takes place and are made available in your callable’s context after the template has been applied. (templated)

Any thoughts?

airflow python
1 Answers

op_kwargs (Optional[Mapping[str, Any]]): This is the dictionary we use to pass in user-defined key-value pairs to our python callable function

templates_dict (Optional[Dict[str, Any]]): This is the dictionary that airflow uses to pass the default variables as key-value pairs to our python callable function. E.g., the 'task_instance' or 'run_id' are a couple of variables that airflow provides us by default. Full list of default variables is here.

So technically, we just need to pass the additional key-value pairs we need through the op_kwargs parameter while using the PythonOperator. I think that's the intended design choice, but we could still use either to pass in our key-value pairs.

Airflow takes care of the rest. The execute method in the PythonOperator merges the kwargs and templates_dict into a single dictionary, which we later unpack in the python_callable function, generally using either **kwargs or **context. Airflow code for this is here.

user_194163140
Related