The Kinds of Enterprise Data

🌐 Also in 28 languages

Español · 日本語 · Português (Brasil) · Français · Deutsch · Italiano · 한국어 · 简体中文 · Português (Portugal) · 繁體中文 · Bahasa Indonesia · Türkçe · Polski · Tiếng Việt · ไทย · Nederlands · العربية · עברית · Українська · हिन्दी · Русский · Svenska · Dansk · Norsk bokmål · Suomi · Español (México) · Français (Canada) · Deutsch (Schweiz)

Image generated with AI

Module 2 of 6 in the chapter Understanding Enterprise Data, chapter 4 of the Data Architect course. Each module stands on its own. Previous: Salesforce Is One System Among Many · Next: The Systems Around Salesforce

The question this module answers: What kinds of data does an enterprise hold, and why do they belong in different places?

A customer record. A list of countries. An order. A field definition. Five years of sales totals. Four million delivery photos. They’re all data, but architecturally they’re not the same thing. They change differently, grow differently, get used differently and are owned by different people, and they shouldn’t automatically live in the same place. So when somebody says “we need this data in Salesforce,” my next question is: what kind of data is it? The answer starts telling me what to do with it.

Start With the Kind, Not the Object

As Salesforce people, we translate requirements into structures almost instantly: customer, Account; photo, File; status, picklist. That’s useful eventually. At the enterprise level, I first want to know what kind of information I’m dealing with. There are many ways to classify it (the DAMA body of knowledge is the standard reference), and I find six kinds useful: master, reference, transactional, metadata, analytical and unstructured. The categories aren’t the architecture. They’re a way to ask better questions.

Master Data: The Things the Business Knows

Master data describes the durable things a business runs around: customers, products, pricing, employees, suppliers, locations, assets. In a Salesforce sales org, that’s your Accounts and Contacts, your products and your price books. At Steve Trucking, a Retail Partner is master data. Keystone isn’t an event; it’s an organization Steve Trucking has an ongoing relationship with. The nine distribution centers are master data too.

Master data outlives the transactions that use it: one Retail Partner can have millions of deliveries. So the first question about it is always who owns it? It may not be Salesforce. The ERP may own products. Salesforce may own prospects while the ERP becomes authoritative once they’re customers. An MDM platform may own the enterprise identity of a customer assembled from several systems. Chapter 6, Master Data and Identity, goes deep. For now, recognize the shape.

Reference Data: The Vocabulary the Business Shares

Reference data is the controlled vocabulary used to classify other data: country codes, currencies, product categories, reason codes, statuses. Steve Trucking’s delivery statuses (scheduled, loaded, out for delivery, arriving, delivered, installed, exception, returned) are reference data. They aren’t deliveries; they’re the words used to describe deliveries.

It looks small and harmless, often a picklist with eight values. Don’t confuse small with unimportant. If Salesforce says Delivered, the ERP says Complete and a Retail Partner sends D, somebody has to define what each means and how they map. Reference data is where integration assumptions hide.

Transactional Data: What Happened

Transactional data records activity: an order placed, a payment made, a delivery completed, a case opened. In the same sales org, that’s quotes and quote lines, orders and order items, payments: each one points back at a customer and a product. Steve Trucking’s million deliveries a year are transactional, and so are their status updates: about eight per delivery, so eight million status changes a year, at a business growing 30% a year.

That’s a completely different growth curve from master data. Three Retail Partners at the top; millions of rows underneath. This is where Steve’s Number of Zeros matters: you can’t look at Retail Partner → Delivery → Status and give every box the same weight. With transactional data I immediately think about volume, velocity, retention, history, archiving, integration, and how people actually need to find it. Part 2 spends a lot of time there.

Metadata: Data About the Data

You’ve been working with metadata all course: an object’s definition, a field’s type, its description, its relationships. The whole enterprise has it: warehouse schemas, integration mappings, data catalogs, API contracts, and for each field an owner, a sensitivity level, a definition, lineage and a retention rule. The values are data; the description of what they mean is metadata. As chapter 3 said, Flag2__c tells nobody anything, and that matters more now that AI reads metadata to work out what data means.

Analytical Data: Data We Made From Other Data

“What’s our on-time rate by Retail Partner over the last five years?” isn’t a transaction. It’s an analytical question, answered by deriving new data from millions of deliveries: on-time percentage, damage rate, revenue per partner, install quality, crew trends. And it’s a different job. The dispatcher asking “what’s the next stop for truck 217?” doesn’t need to scan five years of history. Leadership comparing five years of on-time performance by region does. That’s why analytical data usually lives in a warehouse, a lakehouse or an analytics platform built for that work. That Salesforce can store something doesn’t make the operational platform the place to run every analysis on it.

Unstructured Data: What Doesn’t Fit in Rows

Documents, PDFs, images, audio, video, emails, transcripts, knowledge articles. At Steve Trucking, the delivery photos: about four per delivery, more with installs, four to five million a year. A photo isn’t just another field. It’s big, it can show the inside of someone’s home, it has its own retention rules, and it’s searched differently, if at all.

This kind matters more every year, because AI can now do useful things with information that used to sit outside the structured model. A transcript is data. A PDF manual is data. An install photo is data. That doesn’t mean it belongs in Salesforce’s core. It means it belongs on the landscape.

Click to enlarge

Six kinds, four questions each. The answers change kind by kind, and that’s the point.

The Kind Is Only Half the Question: the Workload

Knowing what kind of data we have isn’t enough. I also want to know what we’re doing with it. Three workloads matter here, and confusing them makes bad architecture.

Operational: run the business. A dispatcher updates a delivery, a crew completes a stop, an agent opens a case. Lots of small reads and writes: find this, update that, show me today’s stops. This is OLTP, online transaction processing, and Salesforce is very good at it; it’s where admins and developers live.

Analytical: understand the business. Every delivery for five years, grouped by partner and DC, compared year over year, joined to finance. Now we’re scanning and aggregating. This is OLAP, online analytical processing, and it’s what warehouses, lakehouses and analytics platforms are built for. Put simply: the org runs the business; warehouses and lakes analyze it. A classic mistake is putting that workload on the operational platform and being surprised as the zeros grow. If someone wants to scan huge amounts of history repeatedly, don’t treat it as “just another report.” Ask whether it’s the right platform. Part 2 comes back to this in Designing for Scale (coming soon) (don’t make core your warehouse), and Part 3 shows how analytical structures are built in Beyond Relational (coming soon).

Streaming: data in motion. Events, telemetry, status changes, sensor readings. Steve Trucking’s trucks send about 200,000 GPS readings a day. The dispatcher cares where truck 217 is now. Does Salesforce need to keep every minute-by-minute reading from every truck? Maybe not. “Where is it now?” and “what were last quarter’s route-efficiency trends?” come from the same source and lead to completely different storage decisions.

How Fast Does It Need to Move?

Then latency. In class I put it this way: real time is waiting at the door for someone to come back. You send them out and stand there, because you can’t carry on until they return. That’s synchronous. Near real time is watching them leave and getting on with something else; when they come back shortly after, you deal with it. That’s asynchronous. And then there’s batch: nobody goes out the door one at a time. The data waits and moves as a group on a schedule: hourly, every few hours, or daily. There are plenty of situations where batch is sufficient, and some where it isn’t. Knowing which is which is the job. Those three aren’t just integration terms; they’re architectural choices.

Take the common ERP integration. A new product is created in the ERP at 11 a.m. Does sales really need to sell it by noon? I’ve seen both answers. Some clients want it as soon as possible; others say, if we haven’t shipped that SKU out to you, we can’t expect you to sell it yet. So products are usually daily, sometimes faster. Customers, at a minimum daily; I’ve seen hourly, and in extreme cases near real time. Orders are different: fulfillment has to start, so near real time. The salesperson closes the deal and moves on, and the order lands in the ERP in seconds to a minute or two. There’s no single right latency for “enterprise data.” Ask what the business actually needs, and don’t make everything real time because it sounds impressive. Real time has a cost.

The Four Questions Behind the Categories

You don’t need to go around announcing “this is transactional data!” The classification is there to trigger four questions about every important data set:

• Who owns it? Which system is authoritative? I break it into three: where is it created, where is it edited, and where is it referenced? Chapter 6 goes deep on these (Created, Edited, Referenced, and How Fast?).
• How fast does it change? Once a year, daily, tens to hundreds of times a second?
• How big does it get? Ten records, ten thousand, ten million, ten billion?
• What’s its lifecycle? Does it live forever, get archived, have to be deleted, lose value over time?

Those four tell me far more than “can Salesforce store this?” And they pull together what we’ve already learned: ownership, lifecycle, the Number of Zeros, where data lives.

One Business Produces Every Kind

At Steve Trucking, the Retail Partner is master data; the status vocabulary is reference data; each delivery is transactional; the definitions of its fields are metadata; the five-year on-time scorecard is analytical; the install photo is unstructured; and the GPS feed is streaming. All of it is “Steve Trucking data.” None of that means it belongs in the same database. That’s the shift.

Beyond the exam: what AI changes. To answer one question, an agent may need a structured customer record, a recent transaction, a PDF manual, a service transcript and live telemetry. Unstructured and streaming data become first-class parts of the architecture even when they don’t belong in the CRM database, and the architect still has to know who owns each source, how current it is, and whether the agent should trust it.

What an Architect Decides Here

• What kind of data each important data set is, and who owns it.
• Whether its workload is operational, analytical, streaming, or a mix.
• How fast it changes, and how fast other systems really need to see the change.
• How volume and lifecycle shape where it lives and for how long.
• Whether Salesforce should own it, hold a copy, reference it, or leave it somewhere better suited to the workload.

When someone hands you a requirement, don’t jump to which object, which field, which API. First ask what kind of data it is, who owns it, how fast it changes, how big it gets, how long it lives, and what we’re trying to do with it. An Account, a delivery event, a five-year aggregate and a photograph can all matter to the same business without being the same architectural problem.

Explore

• DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, 2nd edition (2017): the standard reference for data categories and data management.
• The Steve Trucking brief, section 4 (volumes).