A Decade in Data Engineering Lessons Realities and Where We Go From Here

There was a post on the data engineering subreddit recently that discussed how difficult it is to keep up with the data engineering world.

Did you learn Hadoop, great we are on Snowflake, BigQuery and Databricks now.

Just learned Airflow, well now we have Airflow 3.0.

And the list goes on.

But what doesn’t change, and what have been the lessons over the past decade. That’s what I’ll be covering in this talk. Real lessons and realities that come up time and time again whether you’re working for a start-up or a large enterprise.