Streams and the Lake

Chapter 5 of Data 360 Architect, Part 1: Bringing Data In. Previous: Bringing In CRM Data.

Chapters 3 and 4 opened the doors. This chapter is about what happens just inside them, between the connector and the data model. Data streams are definitely their own animal: they don’t behave like the objects you know from core. The decisions made here are made once, so this is where it pays to slow down.

The positions in this chapter

• The lake is files, a table over them, and a log of every change. That’s why you name things once, why refreshes can be incremental, and why zero copy works.
• The design work is at the stream. Once it exists, there’s little to configure on the data lake object; but the lake object is the stored raw layer, not a temporary stop.
• Pick the category on purpose. Profile is about the person, engagement is activity over time, other is everything else, and none of them can be changed later.
• An engagement stream needs a date that never moves. A date that changes creates duplicate records.
• Failed rows won’t just go away quietly. Read the problem records, and check every first load before you map it.

The path through this chapter

Four modules, each standing on its own.

1. What a Lakehouse Is (coming soon). Parquet files, an Iceberg table, and a log, in plain words.
2. Their Own Animal: the Stream and the Lake Object (coming soon). The flip side, where the design work happens, and what you can’t change.
3. Profile, Engagement or Other (coming soon). Choosing the category, and the event time that never changes.
4. When Rows Don’t Make It (coming soon). Problem records, formula fields, and a first-load checklist.

What an architect decides here

• Each stream’s design: source, fields, types, key, category, event time and name, in the dictionary before it’s built.
• How sources that carry the same data come together: a stream each, combined by a transform, with a key that stays unique across all of them.
• Who watches the problem records, and the checks every first load passes.

Explore

All checked 2026-11-01.

• Data 360 Architecture (opens in a new tab), Salesforce Architects.
• Category (opens in a new tab), Salesforce Help.
• Monitoring Data Stream Problem Records (opens in a new tab), Salesforce Help.

Next: Data Transforms