When the Shape Is Wrong

Module 1 in the chapter Data Transforms, chapter 7 of the Data 360 Architect course. Each module stands on its own. Next: A Batch Transform, Node by Node

The question this module answers: When do you need a data transform, and which kind?

Data comes in through the data streams, lands in the data lake objects, and then we move it to the data model objects: our final structures for identity resolution, segmentation, and a lot of activations. But when you get this data coming in, it might not be in the exact format you need. And there’s a really neat tool for that, called a data transform.

What a transform does

A data transform reads data that’s already in Data 360, reshapes it, and writes the result to another object. You can filter rows, change columns, join, and even do aggregations, and end in a new target object or an existing one. It can go from a data lake object to another data lake object, or from a data model object to another data model object. And you can do much of it without writing code.

Here’s the kind of question that needs one. Take a month of real flights into and out of Phoenix (PHX), as US airlines report them to the US Bureau of Transportation Statistics. Each row is one flight. About half are departures, half are arrivals. Now the business asks: how many flights did each airline land at PHX, each day? No row in the source says that. You need to keep only the flights that really arrived, take the day, and count by airline and day. That’s a transform. We’ll build it in module 2, on the same data you can download from SkyData.

Batch or streaming

When you create a transform, you choose between two kinds, and the difference is the first decision.

Batch. Salesforce calls it “a repeatable series of operations that you can run when data updates.” You run it by hand, or “set it up to run at scheduled intervals.” In a visual builder, you string together nodes: input, filter, join, aggregate, transform, append, output. A batch transform can run the powerful statements, like the aggregations and the group-bys. It works on the whole data set at once.

Streaming. “A streaming data transform reads one record in a source object, reshapes the record data, and writes one or more records to a target object.” It “runs continuously as a streaming process, picking up new or changed data.” It’s written in SQL.

When I first compared them, the streaming transform could only act on individual records. That’s still how it works: one record in, reshaped, out. What’s been added since is joins: a streaming transform can now combine data “from two distinct streams,” using INNER JOIN and LEFT JOIN. So a flight record can pick up its airport’s name as it passes through. But a count by airline and day, across all the rows, is still a batch job.

A rule of thumb, then:

• Whole data set, totals, group-bys, a nightly or hourly result: batch.
• Each record cleaned or enriched as it arrives: streaming.

Things to know about streaming before you build

Salesforce’s considerations page has a few that change how you plan:

• “You can’t change a streaming data transform after saving it.” To change one, you delete it and create another.
• When it’s created, “it can take up to 30 minutes” to start writing to the target; after that, it picks up new or changed records “within a few minutes.”
• “The source and target objects must be different objects.”

Module 5 goes deeper.

Where transforms sit

Remember chapter 6: “A DLO can receive data from at most, one data stream.” But the same page adds that a lake object or a model object “can receive data from one or more data transforms.” That’s how data from several sources comes together in one place: a stream for each source, and transforms that combine them. One rule to plan around: for batch transforms, “You can’t write multiple transforms to the same object,” so the sources come together inside one transform (module 2).

And a transform’s output is just another object. You can use it as the input to the next transform, map it to the model (chapter 8), or build insights on it (chapter 10).

That gives you a way to organize a whole project in layers. In my own aviation project, SkyOrbita, there are three: raw lake objects, exactly what each source sent; enriched lake objects, cleaned, joined and labeled by transforms; and the model objects, mapped only from the enriched layer. The raw layer stays as the source sent it, so you can always load again and trace a value back. It’s an option, not a rule, and it has a cost; module 3 explains why I use it and builds these layers, one transform feeding the next.

Transforms cost credits

Every transform run is processing, and in Data 360, processing consumes credits. Salesforce has a page of billing considerations for data transforms. A batch transform scheduled every hour costs more than one run once a night; a streaming transform runs all the time. And streaming costs many times more per row. On the Data Services rate card (last updated July 2025), a batch transform is 400 credits per million rows and a streaming transform is 5,000. On the Flex Credits card (dated 08.31.2026), batch transforms count as Data 360 Prep at 40 per million rows, and streaming work counts as the Streaming Pipeline at 3,500. So choose streaming when freshness needs it, not by default.

Salesforce’s page of Flex Credits usage types confirms the names: batch transforms count as Data 360 Prep, and streaming transforms as Data 360 Streaming Pipeline, billed on “the higher of the number of rows read or the number of rows written.” Chapter 21 pulls the credit picture together. For now, the same rule as the schedules in chapters 4 and 5: run it as often as the business needs, and no more.

What an architect decides here

• Whether the data needs reshaping in Data 360 at all, or should arrive in the right shape from the source (chapter 3).
• Batch or streaming for each transform, from what it has to do: a whole data set, or each record as it arrives.
• How often each batch transform runs, knowing every run costs credits.
• The layers: what stays raw, what gets enriched, and what the model reads (module 3).

Explore

All checked 2026-11-14.

• Flex Credits Billable Usage Types for Data 360 (opens in a new tab), Salesforce Help (checked 2026-11-14).
• Flex Credits Rate Card (opens in a new tab), Salesforce, dated 08.31.2026 (checked 2026-11-14).
• Salesforce Customer Data Cloud Rate Card (opens in a new tab) (Data Services credits), Salesforce, last updated July 2025 (checked 2026-11-14).
• Batch Data Transforms (opens in a new tab), Salesforce Help.
• Streaming Data Transforms (opens in a new tab), Salesforce Help.
• Considerations for Working with Streaming Data Transforms (opens in a new tab), Salesforce Help.