Available now
When a pipeline fails at 2am.
Every pipeline fails eventually. What matters is how fast you find out why.
This lesson is an incident drill. The nightly job has failed and the dashboard is stale: read the logs, find the row behind the error, set the bad data aside, and plan the reload of the days you missed.
Why data engineers need it
- Being on call is part of most data engineering jobs.
- Bad data rarely raises an error. Logs and checks are how you notice.
- Fixing and backfilling safely is what separates seniors from juniors.
What you will learn
- Reading a pipeline log, starting from the end
- Finding the first error, not the loudest one
- Upstream and downstream: which task to fix
- Tracing an error back to the row that caused it
- Quarantining bad rows instead of deleting them
- Planning a backfill and writing up what happened
The lessons
Nothing to install. Nothing to break.
Every lesson runs in your browser, on your laptop or your phone.