Testing Airflow workflows - ensuring your DAGs work before going into production

Speaker(s): Bas Harenslak
When: (Jul-15 17:00 UTC)

How do you ensure your workflows work before deploying to production? In this talk I’ll go over various ways to assure your code works as intended - both on a task and a DAG level.

In this talk I cover:

  • How to test and debug tasks locally
  • How to test with and without task instance context
  • How to test against external systems, e.g. how to test a PostgresOperator?
  • How to test the integration of multiple tasks to ensure they work nicely together