Airflow metrics with prometheus and grafana

user_14021206 0
Viewed 6707

any one knows how to send metrics from airflow to prometheus, I'm not finding much documents about it, I tried the airflow operator metrics on Grafana but it doesnt show any metrics and all it says no data points.

grafana prometheus airflow
3 Answers

By default, Airflow doesn't have any support for Prometheus metrics. There are two ways I can think of to get metrics in Prometheus.

  1. Enable StatsD metrics and then export it to Prometheus using statsd exporter.

  2. Install third-party/open-source Prometheus exporter agents (ex. airflow-exporter).

If you are going with 2nd approach then the Airflow Helm Chart also provides support for that.

Edit

If you're using statsd exporter here is a good resource for Grafana Dashboard and exporter config.

user_72746170

This is how it worked for me -

  • Running airflow in docker using this doc
  • Added this configuration inside the docker-compose file downloaded in the previous step AIRFLOW__SCHEDULER__STATSD_ON: 'true'
    AIRFLOW__SCHEDULER__STATSD_HOST: statsd-exporter
    AIRFLOW__SCHEDULER__STATSD_PORT: 9125
    AIRFLOW__SCHEDULER__STATSD_PREFIX: airflow
    Under environment section
  • Now run the statsd_export
  • docker run -d -p 9102:9102 -p 9125:9125 -p 9125:9125/udp \ -v $PWD/statsd_mapping.yml:/tmp/statsd_mapping.yml \ prom/statsd-exporter --statsd.mapping-config=/tmp/statsd_mapping.yml
  • Get the statsd_mapping.yml contents from Here
  • Now do docker-compose up to run the airflow and try to run some worflow and you should see logs at http://localhost:9102/metrics
user_67454340

If you installed your Airflow with statsd support:

pip install 'apache-airflow[statsd]'

you can expose Airflow statsd metrics in the scheduler section of your airflow.cfg file, something like this:

[scheduler]
statsd_on = True
statsd_host = localhost
statsd_port = 8125
statsd_prefix = airflow

Then, you can install a tool called statsd_exporter, that captures statsd-format metrics and converts them to Prometheus-format, making them available at the /metrics endpoint for Prometheus to scrape.

There is a docker image available on DockerHub called astronomerinc/ap-statsd-exporter that already maps Airflow statsd metrics to Prometheus metrics.

References:

user_55933170
Related