Have you ever wondered why Apache Airflow builds are asymptotically(*) green? That thrive for “perennial green build” is not magic, it’s the result of continuous, often unseen engineering effort within our CI/CD pipelines & dev environments. This dedication ensures that maintainers can work efficiently & contributors can onboard smoothly.
To tackle the ever growing contributor base, we have a CI/CD team run by volunteers putting in significant work in the foundational tooling. In this talk, we reveal some innovative solutions we have implemented like:
- Handling GitHub Actions pull_request_target challenges
- Restructuring the repo for better clarity
- Slack bot for CI failure alerts
- A cherry picker workflow for releases
- Pre-commit hooks
- Faster website and image builds
- Tackling the new GitHub API rate limits
- Solving chicken-and-egg build issues during releases
Join us to understand the “why” & “how” behind these infra components. You’ll gain insights into the continuous effort required to support a thriving open-source project like Airflow and, hopefully, be inspired to contribute to these areas. (*) asymptotically = we fix failures as quickly as we can when they happen
Amogh Desai
Senior Software Engineer at Astronomer
Jarek Potiuk
Independent Open-Source Contributor and Advisor
Pavan kumar Gopidesu
Technical Lead at Tavant, Member of The Apache Software Foundation & Apache Airflow