From Cron to Assets: Event-Driven Drone Telemetry Ingestion with Airflow

A drone doesn’t care what time it is. It takes off when the mission says so, lands when the battery says so, and uploads its logs whenever the LTE link or WiFi finally cooperates. Cron-based pipelines, by contrast, care deeply about the clock — and that mismatch is where most fleet telemetry stacks quietly bleed money, latency, and engineer sanity on empty polls, half-parsed flights, and workers pinned waiting on slow uploads.

This talk is about throwing the schedule away. In Airflow 3, every completed flight becomes a first-class Asset, and downstream DAGs — parsing, enrichment, anomaly detection, perception-model retraining, regulatory reporting — wake only when the flight they care about actually exists. We’ll cover a custom MAVLinkHook and TelemetryIngestOperator for .ulg and .tlog files, Dynamic Task Mapping across concurrent flights, deferrable sensors, Asset producer/consumer chains replacing ExternalTaskSensor tangles, and honest migration lessons from running old and new DAGs side-by-side on a live fleet.