Scaling Airflow at OpenAI

This talk shares how we scaled and hardened OpenAI’s Airflow deployment to orchestrate thousands of workflows on Kubernetes.

We’ll cover key architecture choices, scaling strategies, and reliability improvements - along with practical lessons learned.