Single Pane Orchestration using Airflow for multiple teams at GoDaddy

As the adoption of Airflow increases within large enterprises to orchestrate their data pipelines, more than one team needs to create, manage, and run their workflows in isolation. With Multi-tenancy not supported natively in Airflow, customers are adopting alternate ways to allow multiple teams to use the same infrastructure. In this session, we will explore how Godaddy uses MWAA to build a Single Pane Airflow set up for multiple teams with common observability platform. This session will help Attendees gain insights the use case, the solution, and its implementation challenges and benefits.

Outline:

• Airflow and MWAA
• Introduction to Multi-tenancy and observability requirements
• Lack of native Airflow support for:
  • Single Pane observablity
  • Multi-tenancy
• Solution and Architecture
  • provide users a single place to monitor their orchestration
  • multiple teams to access their data in isolation
• Implementation Challenges
• Benefits