Running +150 production Airflow on Kubernetes, is that HARD ?

This talk will cover the challenges we can face managing a large number of Airflow instances on private environment.

  • Monitoring and metrics layers for production environment.
  • Collecting and customizing logs.
  • Resource consumption and green IT.
  • Providing support for users and shared responsibility.
  • Pain points