Pro lesson Expert
Land the raw data
Copy the shops' messy till files into raw tables, untouched, with a load you can safely run twice.
20 min11 hands-on steps
In this lesson you will
- Look at the source files before loading anything
- Land every file into a raw table as text, with the file name and load time on each row
- Make the load safe to retry, and reconcile it against what the tills say they sent
Words you'll meet
- Landing folder
- Where source files arrive before anything reads them.
- Wildcard
- A * in a file name that matches any text, so one pattern reads many files.
- Lineage
- Where each row came from: here, the file name and the time it was loaded.
- Manifest
- A list a source sends of the files it delivered, with a row count for each.
The steps
Each step teaches one idea, then has you try it hands-on in your browser, and checks your work.
- 1The job
- 2Open a file first
- 3Let nothing be guessed
- 4Every file at once
- 5Land it
- 6The scheduler retried
- 7Safe to run twice
- 8Reconcile with the manifest
- 9Leave the mess in raw
- 10The customers export
- 11Challenge
This lesson is part of LearnDE Pro
$12 a month or $100 a year: every lesson, spaced-repetition flashcards, and Learn by teaching. Refund on request within 14 days.