Lessons learned for building open source Airflow operators at AWS

In this talk, we’ll share our journey and lessons learned from developing a new open-source Airflow operator that integrates a newly-launched AWS service with the Airflow ecosystem. This real-world case study will illuminate the complete lifecycle of building an Airflow operator, from initial design to successful community contribution.

We’ll dive deep into the practical challenges and solutions encountered throughout the journey, including:

• Evaluating when to build a new operator versus extending existing ones
• Navigating the Apache Airflow Open-source contribution process
• Best practices for operator design and implementation
• Key learnings and common pitfalls to avoid during the testing and release process

Whether you’re looking to contribute to Apache Airflow or build custom operators, this session will provide valuable insights into the development process, common pitfalls to avoid, and best practices when contributing to and collaborating with the Apache Airflow community.

Expect to leave with a practical roadmap for your own contributions and the confidence to successfully engage with the Apache Airflow ecosystem.