콘텐츠로 이동
SteveTechArcSteveTechArc Courses

What a Lakehouse Is

이 콘텐츠는 아직 번역되지 않았습니다.

Module 1 in the chapter Streams and the Lake, chapter 6 of the Data 360 Architect course. Each module stands on its own. Next: Their Own Animal: the Stream and the Lake Object

The question this module answers: Before we work with streams and lake objects, what’s the lake made of, in one page?

In the last three chapters, data came in through the doors and landed in a data lake object. This chapter works with those objects directly, so here’s the lake in short. If you’ve read chapter 2, Under the Hood, this is a recap; that chapter has the depth.

Three parts: files, a table, a log

Section titled “Three parts: files, a table, a log”
  • Files. The data is stored as Parquet files in cloud storage: an open format, stored by column, written once (chapter 2, module 2).
  • A table over the files. Apache Iceberg says which files make up the table right now, and what its columns are. Salesforce adds its own layer on top, and that’s what you see as a data lake object (module 3).
  • A log of every change. A load doesn’t rewrite the table. It writes new files and a new snapshot, and Data 360 tells the work downstream what changed (module 4).
Inside the lake: files, a table, a log The data lake in three layers, from top to bottom. - **At the top:** Flights, a data lake object, the thing you name, map and query. Salesforce adds names and relationships to it. - **In the middle:** the table, in Apache Iceberg format. It holds Snapshot 1, after the first load, kept as history; Snapshot 2, after the second load, highlighted as the table now; and the metadata file, with the schema, the partitions and every snapshot. The metadata file points to the newest snapshot. - **At the bottom:** files in cloud storage, in Parquet format. The first load wrote flights files 1 and 2; the second load wrote flights file 3, and didn't rewrite the old ones. Snapshot 1 uses the first load's files; Snapshot 2 uses files 1 to 3. - **The footnote:** three parts, files in an open format, a table over them and a log of every change, explain why you name it once, why refreshes can be incremental, and why zero copy works. {"author": "Steve Simpson", "blueprint": "1.0.0", "builder": "standard 1.57.1", "built": "2026-11-01", "id": "DGM-D360-SL-01", "rights": "\u00a9 2026 Twinshark LLC. All rights reserved.", "template": "layers 0.1.0", "theme": "stevetecharc 1.4.0"} Flights, a data lake object what you name, map and query names and relationships, added by Salesforce The table (Apache Iceberg) Snapshot 1 after the first load, kept as history Snapshot 2 after the second load: the table now The metadata file the schema, the partitions and every snapshot Files in cloud storage (Parquet) a load writes new files; it doesn't rewrite the old ones FIRST LOAD flights file 1 flights file 2 SECOND LOAD flights file 3 reads the table now points to the newest its files files 1 to 3 Three parts: files in an open format, a table over them, and a log of every change. That's why you name it once, why refreshes can be incremental, and why zero copy works. HOW TO READ IT each load adds files and a snapshot; the table points to the newest the newest load's files © 2026 Twinshark LLC · stevetecharc.com
Click to enlarge

Why it matters in this chapter

Section titled “Why it matters in this chapter”

Three things in this chapter follow from that picture:

  • A lake object is a real table, and the mappings, transforms, insights and segments built on it find it by name. That’s why you name it once (chapter 3).
  • Every load is a snapshot, so a refresh can read only what changed. That’s why incremental refresh is the default, and why it costs fewer credits.
  • Changes come in through the stream. You don’t edit rows in the lake by hand; you change what the stream brings in. Module 4 shows where each fix belongs: at the source, or in a formula field on the stream.

Chapter 2’s rules map lists the rest, from zero copy to real time.

What an architect decides here

Section titled “What an architect decides here”
  • Nothing new to configure. Keep the three parts in mind as you design each stream and lake object in this chapter.

Explore

Section titled “Explore”

All checked 2026-11-14.