These are the sessions we had at Airflow Summit 2024.

Title Speaker(s) Recording Slides

10 years of Airflow: history, insights, and looking forward

Viraj Parekh, John Jackson, Marc Lamberti, Rafal Biegacz, Ash Berlin-Taylor & Elad Kalif

A deep dive into Airflow configuration options for scalability

Ephraim Anierobi

A Game of Constant Learning & Adjustment: Orchestrating ML Pipelines at the Philadelphia Phillies

Mike Hirsch & Sophie Keith

A New DAG Paradigm: Less Airflow more DAGs

Marion Azoulai & Maggie Stark

Activating operational metadata with Airflow, Atlan and OpenLineage

Kacper Muda

Adaptive Memory Scaling for Robust Airflow Pipelines

Cyrus Dukart, David Sacerdote & Jason Bridgemohansingh

AI Reality Checkpoint: The Good, the Bad, and the Overhyped

Maxime Beauchemin

AIP-63: DAG Versioning - Where are we?

Jed Cunningham

Airflow - Path to Industry Orchestration Standard

Rafal Biegacz & Filip Knapik

Airflow 3 - Roadmap Discussion

Madison Swain-Bowden, Kaxil Naik, Michał Modras, Constance Martineau & Shubham Mehta

Airflow and Control-M: Where Data Pipelines Meet Business Applications in Production

Basil Faruqui

Airflow and multi-cluster Slurm working together

Eloi Codina Torras

Airflow as a workflow for Self Service Based Ingestion

Ramesh Babu K M

Airflow at Burns & McDonnell | Orchestration from zero to 100

Bonnie Why

Airflow at Ford: A Job Router Training Advance Driver Assistance Systems

Doug Rogan & Vasantha Kosuri-Marshall

Airflow at NCR Voyix: Streamlining ML workflows development with Airflow

Shahar Epstein

Airflow Datasets and Pub/Sub for Dynamic DAG Triggering

Nawfel Bacha & Andrea Bombino

Airflow UI Roadmap

Brent Bovenzi

Airflow Unleashed: Making Hundreds of Deployments A Day at Coinbase

Jianlong Zhong

Airflow-as-an-Engine: Lessons from Open-Source Applications Built On Top of Airflow

Ian Moritz

Airflow, Spark, and LLMs: Turbocharging MLOps at ASAPP

Udit Saxena

Architecting Blockchain ETL Orchestration: Circle's Airflow Usecase

Nathaniel Rose

Automated Testing and Deployment of DAGs

Austin Bennett

Behaviour Driven Development in Airflow

Ole Christian Langfjaeran

Boost Airflow Monitoring and Alerting with Automation Analytics & Intelligence by Broadcom

Jennifer Chisik

Bronco: Managing Terraform at Scale with Airflow

Jack Cusick

Building in Resource Awareness and Event Dependency into Airflow

Roberto Santamaria, Xiaodong Deng & Anandhi Murali

Building on Cosmos: Making dbt on Airflow Easy

Lewis Macdonald & Ethan Stone

Building Reliable Data Products

Julian LaNeve & Jason Ma

Comparing Airflow Executors and Custom Environments

Deepan Ignaatious

Connecting the Dots in Airflow: From User to Contributor

Briana Okyere, Amogh Desai, Ryan Hatter & Srabasti Banerjee

Converting Legacy Schedulers to Airflow

Fritz Davenport

Customizing LLMs: Leveraging Technology to tailor GenAI using Airflow

Vincent La, Jim Howard & Moulay Zaidane Al Bahi Draidia

DAGify - Enterprise Scheduler Migration Accelerator for Airflow

Konrad Schieban

Data Orchestration for Emerging Technology Analysis

Jennifer Melot

dbt-Core & Airflow 101: Building Data Pipelines Demystified

Luan Moreno Medeiros Maciel

Elevating Machine Learning Deployment: Unleashing the Power of Airflow in Wix's ML Platform

Elad Yaniv

Empowering Airflow Users: A framework for performance testing and transparent resource optimization

Bartosz Jankiewicz

Empowering Business Analysts with DAG Authoring IDE Running 8000 Workflows

Daniil Dubin

Empowering More Teams in your Organization to Self-service their Airflow Needs

Spencer Tollefson

Event-driven Data Pipelines with Apache Airflow

John Jackson

Evolution of Airflow at Uber

Shobhit Shah & Sumit Maheshwari

Evolution of Orchestration at GoDaddy: A Journey from On-prem to Cloud-based Single Pane Model

Ozcan Ilikhan & Amit Kumar

Exploring DAG Design Patterns in Apache Airflow

Sriram Vamsi Ilapakurthy

Featured Workshops

Featured Workshops

From Oops to Ops: Smart Task Failure Diagnosis with OpenAI

Nathan Hadfield

From Tech Specs to Business Impact: How to Design A Truly End-to-End Airflow Project

Taylor Facen

Gen AI using Airflow 3: A vision for Airflow RAGs

Kaxil Naik & Ash Berlin-Taylor

Growing with Apache Airflow: A Providers Journey

Rajesh Bishundeo

Hello Quality: Building CIs to run Providers Packages System Tests

Freddy Demiane, Rahul Vats & Dennis Ferruzzi

How Panasonic Leverages Airflow

Michael Atondo

How the Airflow Community Productionizes Generative AI

Pete Dejoy

How we Run 100 Airflow Environments and Millions of Tasks as a Part Time Job Using Kubernetes

Michael Juster

How we Tuned our Airflow to Make 1.2 million DAG Runs - per day!

Jens Scheffler

How we use Airflow at Booking to Orchestrate Big Data Workflows

Madhav Khakhar, Alexander Shmidt & Mayank Chopra

Hybrid Executors: Have Your Cake and Eat it Too

Niko Oliveira

Integrating dbt with Airflow: Overcoming performance hurdles

Tatiana Al-Chueyr Martins & Pankaj Koti

Investigating the Many Loops of the Airflow Scheduler

Philippe Gagnon

Lessons from the Ecosystem: What can Airflow Learn from Other Open-source Communities?

Michael Robinson

Linkedin's Continuous Deployment

Rahul Gade & Keshav Tyagi

LLMs for Software Development & Apache Airflow

Danny Tarlow

Managing version upgrades without feelings of terror

Daniel Standish

Mastering Advanced Dataset Scheduling in Apache Airflow

Ankit Chaurasia

OpenLineage: From Operators to Hooks

Maciej Obuchowski

Optimize Your DAGs: Embrace Dag Params for Efficiency and Simplicity

Sumit Maheshwari

Optimizing Airflow Performance: Strategies, Techniques, and Best Practices

Pankaj Singh & Pankaj Koti

Optimizing Critical Operations: Enhancing Robinhood's Workflow Journey with Airflow

Kevin Wang, Palanieppan Muthiah & Peiqiu Tian

Orchestrating & Optimizing a Batch Ingestion Data Platform for Americas #1 Sportsbook

Gunnar Lykins

Orchestration of ML workloads via Airflow & GKE Batch

piotr-lesniak & Rafal Biegacz

Overcoming Custom Python Package Hurdles in Airflow

Amogh Desai & Shubham Raj

Product Management perspective on Data Observability with Databand

Steve Sawyer

Profiling Airflow tasks with Memray

Cedrik Neumann

Refactoring DAGs: From Duplication to Delightful Efficiency with a Centralized Library

Gil Reich

Running Airflow Tasks Anywhere, in any Language

Ash Berlin-Taylor & Vikram Koka

Scalable Development of Event Driven Airflow DAGs

Ipsa Trivedi & Subramanian Vellaiyan

Scale and Security: How Autodesk Securely Develops and Tests PII Pipelines with Airflow

Bhavesh Jaisinghani

Scaling AI Workloads with Apache Airflow

Shubham Mehta & Rajesh Bishundeo

Scaling Airflow for Data Productivity at Instacart

Anant Agarwal

Security United: collaborative effort on securing Airflow ecosystem with Alpha-Omega, PSF & ASF

Michael Winser & Jarek Potiuk

Seeing Clearly with Airflow: The Shift to Data-Aware Orchestration

Constance Martineau & Tzu-ping Chung

Simplified user management in Airflow

Vincent Beck

Streamline data science workflow development using Jupyter notebooks and Airflow

Neha Singla & Sathish kumar Thangaraj

Streamlining a Mortgage ETL Pipeline with Apache Airflow

Zhang Zhang & Jenny Gao

Streamlining DAG Creation with YAML in Large Organizations

Howie Wang

Stress-Free Airflow development: From Dev to Prod at Stripe

Nick Bilozerov & Thomas Tauber-Marshall

The Essentials of Custom Executor Development

Dennis Ferruzzi & Syed Hussain

The road ahead: What’s coming in Airflow 3 and beyond?

Vikram Koka

The Silent Symphony: Keeping Airflow's CI/CD and Dev Tools in Tune

Jarek Potiuk

Unleash the Power of AI: Streamlining Airflow DAG Development with AI-Driven Automation

Sriharsh Adari, Jeetendra Vaidya & Joseph Morotti

Unlocking FMOps/LLMOps with Airflow: A guide to operationalizing and managing Large Language Models

Parnab Basak

Unlocking the Power of AI at Ford: A Behind-the-Scenes Look at Mach1ML and Airflow

Elona Zharri, Nikhil Nandoskar & Prince Bose

Unlocking the Power of Airflow Beyond Data Engineering at Cloudflare

Jet Mariscal

Using Airflow operational data to optimize Cloud services

Olivier Daneau

Using the power of Apache Airflow and Ray for Scalable AI deployments

Venkata Jagannath & Marwan Sarieddine

Weathering the Cloud Storms With Multi-Region Airflow Workflows

Amit Chauhan

What If...? Running Airflow Tasks without the workers

Wei Lee

Why Do Airflow Tasks Fail? An Analysis through Machine Learning Techniques

Julian LaNeve & David Xue

Winning Strategies: Powering a World Series Victory with Airflow Orchestration

Oliver Dykstra

Workshop: DAG writing best practices

Marc Lamberti, Ryan Hatter & Michael Robinson

Workshop: Get Airflow Certified

Marc Lamberti & David Patlut

Workshop: Leveraging Automation Analytics & Intelligence with Airflow for Enhanced Workflow Observability

Chetan Kapoor

Workshop: Orchestrating Business Services with Airflow and Control-M

Minh Nguyen & Joe Goldberg

Workshop: Orchestrating ML Workflows with Amazon MWAA

Sriharsh Adari, Aneesh Chandra PN & Anurag Srivastava

Workshop: Unleash Airflow's Potential with hands-on Performance Optimization

Aneel Murari & Jimmy Martinez

Workshop: Your First Apache Airflow Contribution

Ryan Hatter