What Data 360 Sits On

Module 1 in the chapter Under the Hood: What Data 360 Sits On, chapter 2 of the Data 360 Architect course. Each module stands on its own. Next: Parquet: Data Stored by Column, Written Once

The question this module answers: Why does Data 360 need a different kind of platform, what is it built on, and why should an architect care?

Chapter 1 showed the screens and the flow: data comes in, lands in the lake, gets mapped to the model, and goes back out. This class is for architects and aspiring architects, and it goes deeper than what an admin or a developer usually needs. So before we build anything, I want to open the covers. Not to make you a data engineer. To make you the person in the room who can explain why Data 360 behaves the way it does.

Because a lot of what surprises people in Data 360 isn’t a Salesforce whim. You can’t change a field’s type. Data comes in as whole loads, not one edited row at a time. Refreshes are incremental. Real time is a separate thing with its own limits. Data in Snowflake can be read without copying it. Every one of those comes from what’s underneath. Once you see the layers, the rules stop being rules to memorize.

Here’s why I start here. Without knowing what it’s built on, people see only the negatives. Why can’t I change an API name? Why is deleting data so hard? Why can’t I update one record, the way I can in core? They expect core’s capabilities, and Data 360 isn’t core. It’s built on a whole different stack, for a whole different purpose. Know the power of what it’s built on, and you can accept its limits, understand how it behaves, and work with it instead of against it.

First, why not something we already have?

Before we look at what Data 360 sits on, let’s ask the question every architect should ask of a new platform: why couldn’t we do this with what we already have? Ruling out the familiar options is the best way to see why Salesforce went to a different, more modern stack.

Think about the job first. Data 360 brings together every system’s view of every customer: the CRM orgs, the website clicks, the order history, the files from partners, the warehouse tables. Years of it, often billions of rows. Then it matches people across all of that, adds things up, and picks audiences out of it. That’s a very different job from saving one customer’s address.

Why not Salesforce core?

Core is the best place I know for the records people work with all day: the account, the case, the opportunity. I like to think of core as the front of your office, where the work happens. It’s built for transactions: one record at a time, opened, edited and saved, with sharing, automation and validation around every change.

That design has a price at Data 360’s scale.

• Storage is sized for working records. Salesforce allocates Enterprise, Performance and Unlimited orgs “10 GB for data storage, plus incrementally added user storage”, and most records take “approximately 2 KB”. Do the math: 10 GB is about five million records. A large company’s website can produce that many clicks in a day.
• Every record carries the weight of a transaction. Triggers, flows, sharing and limits are exactly what you want around an order someone is editing. They’re overhead around a billion web events nobody will ever edit.
• Big volumes need special care. The Data Architect course has a whole chapter on large data volumes in core (Designing for Scale: When Volume Changes Everything). Core can hold a lot, with design work. It isn’t built to scan years of history across every source.

So history and high-volume data belong somewhere built for them, with only what people work on kept in core. That’s not a criticism of core. It’s using each tool for its job.

Why not a relational database?

A relational database (the kind you query with SQL: Oracle, SQL Server, PostgreSQL) is still the best design I know for business transactions. But a classic transactional database stores data row by row and is built to find, lock and change a few rows quickly and safely. Ask it “the average order value by region over five years” and it reads every column of every row to answer.

The industry’s answer to that was the data warehouse, built for analysis. Salesforce’s own guide tells the history: first, “operational/OLTP databases”; then “data warehouses OLAP/Big Data platforms that were primarily used for analytical processing and remained relational and centralized.” OLTP means transactions; OLAP means analysis. Warehouses solved the analysis, but they kept the data in their own place and their own format. Salesforce’s view: “operations platforms and analytical platforms stayed separate”, and the big data platforms “serve as data silos.”

Why not NoSQL?

NoSQL databases (key-value stores and document databases, for example) were built for a different problem: huge numbers of small, fast reads and writes by key, with flexible shapes. Give me the shopping cart for this session ID, now. They’re excellent at that.

But they’re built to fetch a known item, not to join, add up and compare everything across many sources. Matching a person across ten systems, then counting their orders over three years, isn’t a lookup by key.

So: an open lakehouse, plus a real-time layer

Here’s where that leaves us. The job needs:

• History and volume at low cost, which is cloud object storage.
• Analysis across everything, which is columnar files and big parallel engines.
• Data that others can read without copying, which is open formats.
• A fast answer about one person, now, which is a separate real-time layer: memory, backed by a large, fast store on solid-state drives (module 5).

That’s what Salesforce built. Its guide describes the platforms of the future as “unified, flexible, accessible, and open”, and notes that the big data platforms “are converging on adopting open file formats (Parquet) and table formats (Iceberg).” Data 360 sits on exactly those. The rest of this chapter opens them up, one layer at a time.

Click to enlarge

The stack, bottom to top

Salesforce’s architecture guide describes Data 360 in layers. Here they are, from the bottom up, in plain words.

1. Cloud storage, on Hyperforce. At the very bottom are ordinary files in cloud object storage, such as Amazon S3 or Azure Blob storage. Data 360 “runs on Hyperforce”, Salesforce’s cloud foundation, deployed on the big public clouds. In Salesforce’s words, in cloud data lakes “the fundamental storage unit is a file.”

2. Parquet files. Those files are in an open format called Apache Parquet. Salesforce: “storage consists of data lake files… in Parquet format.” Parquet stores data column by column, and a file is written once and then read many times. Module 2 is all about it.

3. Iceberg tables. Files alone aren’t a table. Apache Iceberg is an open table format that says which files make up a table right now, what its columns are, and how it changed over time. Salesforce: “Iceberg manages a metadata file for each table, detailing schemas, partition specs, and snapshots.” Module 3.

4. Salesforce’s own table layer. On top of Iceberg, Salesforce adds what it calls the cloud table: “semantic metadata such as column names and relationships”, plus settings like target file size and compression. This is where a table becomes a data lake object you see in Data 360.

5. The change feed. Every change to an Iceberg table becomes an event, and Data 360 keeps a feed of exactly what changed. That’s why the platform works incrementally. Module 4.

6. The engines. Two engines do the work over the lakehouse: Spark for processing (transforms, insights, segments) and Hyper for fast SQL queries. Module 5.

7. Beside it all: the real-time layer. A lakehouse is built for large amounts of history, not for answering about one person in milliseconds. So Salesforce built a second layer: memory, plus a fast SSD store it calls the low latency store. Module 5 too.

8. On top: the screens in core. And at the top, what you saw in chapter 1: the Data 360 app in your home org. The screens live in core; everything under them lives in the tenant.

Click to enlarge

Two layers, not one

Salesforce describes the storage as “a tiered but integrated storage model that combines the strengths of a lakehouse with real-time storage.” The lakehouse holds “large volumes of historical and batch data.” The real-time storage “is optimized for low-latency access and high-frequency updates.”

Hold on to that sentence. It’s the reason almost everything in Data 360 runs in minutes or hours, and the reason real time is a separate design with separate limits (chapter 13). Two layers, built for two different jobs.

Why open formats matter

Parquet and Iceberg aren’t Salesforce inventions. They’re open-source projects of the Apache Software Foundation, used across the industry. Salesforce’s own guide says the big data platforms “are converging on adopting open file formats (Parquet) and table formats (Iceberg) enabling data federation (data in) and sharing (data out).”

That’s a big deal for an architect. It means your Data 360 lake is built from the same materials as your company’s Snowflake, Databricks or BigQuery lake. They can read each other’s tables where they sit. That’s what zero copy is (chapter 15), and it works because of what’s down here.

It also means this knowledge lasts. Product names change; Data Cloud became Data 360. Parquet and Iceberg are industry standards your data engineers already use. When the warehouse team says “our lake is Iceberg on S3”, you’ll know what they mean, and what it means for Data 360.

How deep we’ll go

Deep enough to explain behavior, no deeper. You’ll learn what each layer is, what it’s good and bad at, and which Data 360 rule it explains. You won’t learn file encodings, page sizes or how to tune a Spark job. That’s data engineering, and Data 360 manages it for you. Module 6 ties it all together: every “why can’t I…” in this course, traced back to one of these layers.

What an architect decides here

• Nothing to configure. This is the picture you’ll use to explain every later decision: to your team, to the business, and to the data engineers on the other side of a zero copy connection.

Explore

All checked 2026-11-14.

• Data 360 Architecture (opens in a new tab), Salesforce Architects: the storage model, the lakehouse layers, the low latency store, the compute fabric.
• Apache Parquet (opens in a new tab) and Apache Iceberg (opens in a new tab), the Apache Software Foundation.
• Data and File Storage Allocations (opens in a new tab), Salesforce Help: core’s data storage.
• Salesforce Data Storage: Estimated Record Size by Object Type (opens in a new tab), Salesforce knowledge article 000383664: about 2 KB a record.
• Where Relational Stops (coming soon), the Data Architect course: when to choose a shape other than relational.