the Integration Log

Module 2 of 9 in the chapter Reliability, Recovery, and Observability, chapter 14 of the Integration Architect course. Each module stands on its own. Previous: Every Arrow, Out and In · Next: The Log Is Data: Retention and Masking

The question this module answers: Where is every integration call recorded, and how do you design that record so it’s there when you need it?

I Knew When It Happened

On one project, the middleware team didn’t call Salesforce’s standard APIs. They called custom APIs that we built for them. And because every call came through our own door, I was able to timestamp every incoming call. I knew when it happened. I knew how long it took. I even recorded the payload of what they passed in.

Every call went to one custom object, the integration log. We wrote to it asynchronously, so it didn’t slow down the transaction, and it let us track everything and replay it. We even passed a unique identifier on every request, so during load testing I could match our log, line by line, against the middleware team’s performance logs. The Data Architect course tells this story too, from the data side (Proof of Delivery).

This module is the build: what goes in an entry, how one ID ties the layers together, how to keep a log that doesn’t vanish with the failure it’s recording, and what Salesforce itself recommends once the numbers get big.

Standard Door or Your Own

Notice where that story started: custom APIs. That’s not a detail.

When another system calls Salesforce’s standard REST API, it writes straight into your objects. There’s no code of yours in the middle, so there’s nowhere for you to write a log entry. Salesforce records the API calls in its own event logs (module 7), but your own integration log only sees what passes through something you built: an Apex REST service, a flow, a trigger, a platform event subscriber.

So it’s a trade-off, and you decide it in the first week or two of a project, in design. With the standard API, the other team needs to know your objects and fields well, and the integration layer has to keep the log. With your own door, you get the log and a contract that hides your inner workings, and you need good Apex developers to build it (chapter 8 module 5).

Either way, someone keeps the log. The question is who, and where.

What One Entry Holds

Whatever writes it, a log entry needs enough to answer the questions from module 1 later, when someone is looking at a failure at two in the morning.

• Which integration, by a name everyone uses: Redrock manifests in, not Integration 3.
• Which direction: in or out.
• When it started and how long it took.
• The ID that travels with it, the same one every layer records (next section).
• The business key: the Work Order, the partner’s reference, whatever a person would search for.
• How it ended: the status code, the error, and whether it will be retried.
• The payload, or as much of it as you’re allowed to keep. That last part matters, and it’s the whole of module 3: a log that holds personal data is personal data.

On that project I used a long text field for the payload, because the payloads weren’t that large. Check yours. A log is no help if half the payload was cut off.

One ID, Through Every Layer

Here’s the thing I learned from matching those load-test logs. Every layer keeps its own log. Salesforce has one, the integration layer has one, the other system has one. On a bad day, you need to put them side by side. Without a shared ID, that’s guesswork by timestamp. With one, it’s a search.

So decide, for every arrow, which ID travels the whole way, and make every layer write it down.

On Steve Trucking’s status arrow, there’s a natural one. When a delivery’s status changes, Salesforce publishes an event, and every event message carries a unique ID, its EventUuid (chapter 11 module 4). The integration layer records that ID when it receives the event, and passes it on when it calls Redrock’s webhook, in the webhook-id header that the Standard Webhooks shape gives every message (chapter 8 module 6). Three logs, one ID.

Click to enlarge

One ID, three logs.

That’s what “top to bottom” means for logging. Not one log to rule them all, but one ID that lets you read them together.

The Log That Rolls Back

Now a trap, and it catches good developers.

Say your Apex code processes an incoming manifest. Something fails, an exception is thrown, and in the catch you write an error record to your integration log. Then the exception goes on up, and the transaction fails. The Apex guide is plain about what happens next: “If the request doesn’t complete successfully, all database changes are rolled back.” All of them. Including your error record. The failure happened, and the only evidence of it was undone with it.

There are two ways out, and Salesforce has given you both.

Publish the log entry as a platform event, with Publish Immediately. Normally I’d steer you to Publish After Commit, so nobody hears about a save that didn’t happen (chapter 11 module 3). A log entry is the exception. Salesforce’s guide says to choose Publish Immediately “if the event publishing isn’t tied to transaction data”, and that it “publishes the event message when the publish call executes, regardless of whether the transaction succeeds.” That’s exactly what you want from a log. A subscriber then writes the event to the log object, in its own transaction. And it’s asynchronous, which is how we kept the log from slowing down the transaction in the first place. Remember the events count toward your event allocations (chapter 11 module 5).

Write the payload to a big object with Database.insertImmediate(). Big objects work differently. Salesforce’s own playbook puts it this way: “Because Database.insertImmediate() commits the big object record immediately — outside the normal Apex transaction, so it can’t be rolled back — the payload survives even if processing throws later.”

Outbound calls have their own rule, from chapter 9: call, then write. Salesforce won’t let you make a callout after uncommitted work in the same transaction, so the log entry for a callout is written after the answer comes back (chapter 9 module 5).

And in a flow, the fault path is where the log is written. Salesforce’s patterns guide says a flow’s default failure email “should be enhanced to put in custom behavior such as creating a case or writing errors to a custom object that can be monitored and tracked.” Many teams publish an error platform event from the fault path, and let one flow pick them all up and send the alerts. It’s the same idea: get the record out of the transaction that’s failing.

Click to enlarge

A log has to outlive the failure it records.

At Volume: the Ledger and the To-Do List

A custom object log is the right start, and for many integrations it’s all you’ll need. But do the Number of Zeros before you commit to it.

Steve Trucking’s deliveries change status about 26,400 times a day. If every status event that Salesforce publishes is recorded, that’s about 9.6 million entries a year, every year. Then add the manifests, and every other arrow.

In 2026, Salesforce published a pattern for exactly this, and it splits the log in two (The Big Objects Playbook: Payload Capture and Replay (opens in a new tab)).

The ledger is a big object. A copy of every event is written to it before any business logic runs. Big objects are built for hundreds of millions of records, and in the playbook’s words they “are not transactional database tables, instead they are a durable, queryable ledger.” Every payload goes in, success or failure.

The to-do list is a custom object. Only the failures go here, small records with the error, the status, a retry count, and a pointer back to the payload in the ledger. This is the object your support team works from, with list views, reports and flows.

The two are linked by the event’s ID. When a failure needs to be replayed, the retry reads the to-do list, fetches the payload from the ledger by that ID, and runs it again (module 6). The playbook names them well: the ledger is “the filing cabinet that permanently stores everything”, and the custom object is “the daily to-do list for your support team”.

Click to enlarge

The filing cabinet and the to-do list.

Before you choose a big object, know what you’re giving up, because the platform is honest about it:

• The index is forever. Every query has to filter on the big object’s index, in order. So design the index around the question you’ll ask most often, usually “give me the payload for this event.” And get it right the first time: “After you’ve deployed a big object, you can’t edit or delete the index. To change the index, start over with a new big object.”
• No platform automation, no standard screens. No triggers, no flows, no list views, no standard reports. People see the ledger through a Lightning web component you build.
• No transactions, and writes can partly fail. The guide says so: “Big objects don’t support transactions”, and the best practice when writing is “to have a retry mechanism in place.”
• No encryption. Encrypted data copied into a big object is “stored as clear text on the big object.” For a log full of payloads, that’s a design question, and module 3 answers it.
• The allocation. Salesforce Help says orgs are “allocated storage for 1 million big object records per org.” At 26,400 a day, Steve Trucking’s ledger passes that in about 38 days. So a ledger that size means more big object capacity, or a shorter retention, or keeping the full ledger in the integration layer and only the failures in Salesforce. That’s an architect’s call, made with the numbers, not discovered later.

Test knowledge. Know how to log an integration so the log survives a failure: a platform event with Publish Immediately is published “regardless of whether the transaction succeeds”, and Database.insertImmediate() writes to a big object outside the transaction. Know big objects for logs at volume: insert-only, queried by their index (which can’t be changed after deployment), no triggers or flows, no standard UI, 1 million records allocated per org.

Steve Trucking’s Logs

Back to the failure plan from module 1, and its first column: where is each arrow logged?

Arrow	Salesforce	The integration layer	Shared ID
Manifests in from Redrock	Nothing of its own: standard API upserts, seen in Salesforce’s event logs	Redrock’s call, and its own upsert into Salesforce	Redrock’s reference
Status events out	The ledger: every event it publishes	Each event received	EventUuid
Status out to Redrock	Nothing: Salesforce doesn’t make this call	Every attempt and its answer	EventUuid, sent as webhook-id
Failures, any arrow	The to-do list, a custom object, for anything Salesforce must act on	Its own error queue	the same ID

Two things to notice. First, a lot of the logging lives in the integration layer, because that’s where most of the calls are made. Salesforce can only log what passes through its own code. Second, the shared-ID column is what makes the table work. Without it, these are four separate logs. With it, they’re one story.

Beyond the exam: what AI changes. Point an AI assistant at three logs with a shared ID, and it can find the one status that never arrived, among millions of lines, in a moment. Without the shared ID, it guesses by timestamp, just as you would. The design decision that makes AI useful here is the same one that makes your own search work.

What an Architect Decides Here

• Who keeps the log for each arrow, decided in design: your own door logs in Salesforce; the standard API means the integration layer logs.
• What every entry holds, and the one ID every layer writes down.
• How a log survives the failure it records: a Publish Immediately event, insertImmediate() to a big object, the fault path in a flow, and “call, then write” for callouts.
• Where the payloads live at volume: a custom object, a big object ledger beside a to-do list, or the integration layer, sized with the Number of Zeros and the allocation.
• The big object’s index, designed around the question you’ll ask most, before it’s deployed.

Explore

• Salesforce Developers Blog, The Big Objects Playbook: Payload Capture and Replay (opens in a new tab): the ledger and the to-do list, step by step (June 2026, checked 2026-11-07).
• Big Objects Implementation Guide, Big Objects Best Practices (opens in a new tab) and Define and Deploy Custom Big Objects (opens in a new tab): limits, transactions, encryption, the index (checked 2026-11-07).
• Platform Events Developer Guide, Platform Event Fields (opens in a new tab): Publish Immediately and Publish After Commit (checked 2026-11-07).
• Apex Developer Guide, Transaction Control (opens in a new tab): what rolls back (checked 2026-11-07).
• Proof of Delivery: the Data Architect course, the integration log from the data side.