Data Transforms

Chapter 6 of Data 360 Architect, Part 1: Bringing Data In. Previous: Streams and the Lake.

Data comes in through the streams and lands in the lake objects, and from there we map it to the model. But when it comes in, it might not be in the exact format you need. There’s a really neat tool for that, called a data transform. It reads data that’s already in Data 360, reshapes it, and writes the result to another object, and for most of the work you don’t write code.

The positions in this chapter

• Batch for the whole data set, streaming for each record. Totals, group-bys and joins across everything are batch; cleaning or enriching each record as it arrives is streaming.
• Build to the end before you save. A batch transform needs an output node; a streaming transform can’t be changed once it’s saved.
• The row decides the key. An aggregate’s key is built from what it groups by, with IDs, a separator and a fixed format.
• Know what each run costs. You can’t choose incremental or full; the transform’s steps decide, and an aggregation always runs in full.
• The builder first, code when it can’t do the job.

The path through this chapter

Five modules, each standing on its own.

1. When the Shape Is Wrong (coming soon). What a transform does, batch or streaming, and where transforms sit.
2. A Batch Transform, Node by Node (coming soon). Arrivals by airline and day: input, filter, transform, aggregate, output.
3. A Key That Holds (coming soon). The output’s primary key, and what the fully qualified key is for.
4. Streaming Transforms (coming soon). One SQL statement: UNION, joins, and transforms that share objects.
5. Beyond the Builder (coming soon). Flattening JSON, pivots, incremental runs, write modes and Code Extension.

What an architect decides here

• For each reshaping, batch or streaming, from what it has to do.
• The grain and the key of every output: one row per what, and the columns its key is built from.
• How often each batch transform runs, knowing which kind of run it gets and that every run uses credits (chapter 20).
• When the builder isn’t enough, and who owns the code that replaces it.

Explore

All checked 2026-11-01.

• Batch Data Transforms (opens in a new tab), Salesforce Help.
• Streaming Data Transforms (opens in a new tab), Salesforce Help.
• Batch Data Transform Run Types (opens in a new tab), Salesforce Help.

Next: From the Lake to the Model (coming soon)