One Orchestrator for Data & MLOps: Scaling 400+ Pipelines on Managed Airflow at Waze

Operating data workflows at scale often forces teams to juggle separate orchestrators for analytics and machine learning. In this session, Waze team share how Waze unified data pipelines and MLOps onto one platform.

We will cover:

1. Operating model & developer self-service: platform infra supporting 400+ DAGs, custom Airflow plugins (BigQueryDataArrivalSensor), and utomated CI/CD.

2. Gen 3 migration: moving from upgrade friction to seamless operations on Managed Airflow (Composer 3).

3. Workload tuning: practical lessons on deferrable operators and DAG parsing efficiency across high-frequency schedules.

4. AI-driven impact: scheduling AI optimization workflows that slashed BigQuery pipeline workloads by up to 95%.