Testing Airflow workflows - ensuring your DAGs work before going into production
Speaker(s): Bas Harenslak
When: (Jul-15 17:00 UTC)-----
How do you ensure your workflows work before deploying to production? In this talk I’ll go over various ways to assure your code works as intended - both on a task and a DAG level.
In this talk I cover:
- How to test and debug tasks locally
- How to test with and without task instance context
- How to test against external systems, e.g. how to test a PostgresOperator?
- How to test the integration of multiple tasks to ensure they work nicely together