Testing Airflow workflows - ensuring your DAGs work before going into production

Speaker(s): Bas Harenslak
When: (Jul-15 17:00 UTC)
-----

How do you ensure your workflows work before deploying to production? In this talk I’ll go over various ways to assure your code works as intended - both on a task and a DAG level. In this talk I will cover:

  • How to test and debug tasks locally
  • How to test with and without task instance context
  • How to test against external systems, e.g. how to test a PostgresOperator?
  • How to test the integration of multiple tasks to ensure they work nicely together