Pro lesson Expert
Warehouses, lakes and lakehouses
Compare the architectures companies actually run, see why a folder of files is not a table, and pick the right one for the job.
10 min8 hands-on steps
In this lesson you will
- Say what a warehouse, a lake and a lakehouse each give you, and what each costs
- See why a folder of files is not a table, and how a manifest of snapshots fixes it
- Read a table as it was in the past with time travel
- Choose an architecture by size and need, not by hype
Words you'll meet
- Data warehouse
- A managed database for analytics: you load data in, and it gives you fast SQL and safe, all-or-nothing changes.
- Data lake
- Files in cheap storage, in any format, that many tools can read. Just files: no tables, no all-or-nothing changes.
- Lakehouse
- A lake plus a table format, so files in cheap storage behave like warehouse tables.
- Table format
- A log kept next to the files that says which files make up the table, like Apache Iceberg or Delta Lake.
- Snapshot
- One version of a table: the exact list of files that made it up after one change.
- Manifest
- The list of files in a snapshot.
- Time travel
- Reading a table as it was at an earlier snapshot.
- Medallion
- Naming the layers bronze (raw), silver (clean) and gold (modeled).
The steps
Each step teaches one idea, then has you try it hands-on in your browser, and checks your work.
- 1A warehouse
- 2A lake
- 3Which files are current?
- 4A table format
- 5Time travel
- 6Commit a snapshot
- 7Bronze, silver, gold
- 8Challenge
This lesson is part of LearnDE Pro
$12 a month or $100 a year: every lesson, spaced-repetition flashcards, and Learn by teaching. Refund on request within 14 days.