Apache Airflow/Composer : how to connect to https using http connector with untrusted certificate

Viewed 313

I am looking to connect to an external API through HTTPS with Airflow.

To do that, I configure my http Airflow connector regarding the documentation.

I setted my host with my url : myurl.com Then i setted the schema value to 'https' as expected in the documentation.

And because I need a certificate, according to the http_hook documentation (airflow doc) I added extra option : {"cert":"/home/airflow/gcs/data/mycertificate.pem"}

I am using Composer on the Google Cloud Platform, so I putted my certificate in the corresponding google cloud storage bucket (doc)

I am not sure about my configuration but when I try to use the SimpleHttpOperator

get_token = SimpleHttpOperator(
    task_id='get_access_token',
    method='POST',
    headers={
        "Authorization": "Basic mytooken=="},
    endpoint='/SASLogon/oauth/token',
    http_conn_id='myconnid',
    trigger_rule="all_done",
    response_filter=lambda response: response.json()['access_token']
)

get_token

I have the following error :

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)

I am not sure if the error is due to my cert which is not found or due to my self-signed certificate.

Anyway, I am pretty sure I have got to add a configuration somewhere because my certificate is self-signed.

Have you got any idea ?

0 Answers
Related