Lineage and Disaster Recovery procedures for Airflow based on Cloud Composer & Google Cloud Platform

Speaker(s):
Filip Knapik
Filip Knapik
Michał Modras
Michał Modras
Rafal Biegacz
Rafal Biegacz
Bartosz Jankiewicz
Bartosz Jankiewicz
Leah Cole
Leah Cole
Victor Aoqui
Victor Aoqui
Arun Joy Vattoly
Arun Joy Vattoly
By Filip Knapik Michał Modras Rafal Biegacz Bartosz Jankiewicz Leah Cole Victor Aoqui Arun Joy Vattoly

Hands on workshop showing how easy it is to use Airflow in a public cloud (based on Cloud Composer).

Workshop consists of 3 parts:

  • Best practices for Airflow DAG development for Airflow
  • Emitting data lineage information for Airflow
  • Disaster Recovery routines for Airflow

This workshop is mostly targeted at medium and advanced Airflow users who would like to learn more about Airflow and Cloud Composer and how to use the features like data lineage to enhance data observability and disaster recovery procedures to have a response in case of disaster recovery events.