Airflow Already Knows the Root Cause

Dashboards show symptoms. Lineage shows relationships. But neither explains why something happened.

In the AI era, Airflow is no longer just an orchestrator - it’s a source of operational intelligence. OpenLineage, the open standard for data lineage, emits rich, structured context across Airflow, Spark, and other tools - including failure logs and detailed execution metadata.

With recent improvements - out-of-the-box run ID correlation between Airflow entities, powerful hook-level lineage for SQL operators (capturing queries and query IDs from within Python operators), human-in-the-loop metadata (who approved what and when), and more - the context layer is more powerful than ever.

In this talk, you’ll learn what OpenLineage provides out of the box today, what’s coming next, and how this foundation can power AI-driven assistants, auditors, or operational agents.

I’ll demo the Astro Observe Investigation Agent to illustrate the possible impact - but the focus is the context layer already available to everyone.