Choosing the Size of the Truck
Module 7 of 8 in the chapter Moving Data at Scale, chapter 10 of the Integration Architect course. Each module stands on its own. Previous: Did It All Arrive? · Next: Keeping a Copy in Sync
The question this module answers: How do you decide a feed needs the loading dock, and how do you check it still fits in three years?
Count the Records in One Trip
Section titled “Count the Records in One Trip”In module 1, the teller window takes 200 records at a time and sends you to the back of the line. A million records is 5,000 trips. The loading dock takes the whole truck, and a million records is a handful of calls. The question is where one ends and the other begins.
Salesforce draws the line in its own words: “Any data operation that includes more than 2,000 records is a good candidate for Bulk API 2.0… Jobs with fewer than 2,000 records should involve ‘bulkified’ synchronous calls in REST (for example, Composite) or SOAP” (Bulk API and Bulk API 2.0 Limits and Allocations (opens in a new tab), Salesforce Developer Limits and Allocations Quick Reference, checked 2026-11-01).
Notice the words: “a good candidate.” It’s a guide, not a wall. The count is the first question, and it’s the one I teach with my Number of Zeros: how many records in one trip, today and at year three. The second question is what the sender needs back. The teller window gives an answer for every record, straight away. The dock gives a job ID now and the results later. A feed that needs the answer now leans to the teller, even a bit past 2,000. A feed that can wait for its results leans to the dock.
The Dock’s Own Constraints
Section titled “The Dock’s Own Constraints”Chapter 6 ended with a promise: every mechanism chapter gives its own limits, with the release they were checked against, so you can match them against your numbers (Constraints). Here are the loading dock’s, for Bulk API 2.0 ingest jobs (Limits and Allocations, and the Bulk API 2.0 and Bulk API Developer Guide (opens in a new tab), Winter ‘27, both checked 2026-11-01):
| What | The limit |
|---|---|
| Records uploaded | 150,000,000 in a rolling 24 hours |
| One upload | 150 MB after Salesforce encodes it, so about 100 MB of CSV |
| Batches | 15,000 in a rolling 24 hours, shared with Bulk API 1.0; on 2.0 Salesforce makes them, and only ingest jobs use them |
| How long a job can stay open | 24 hours |
| Each transaction | 200 records; CPU time on Salesforce’s servers up to 60,000 milliseconds, a limit of its own for bulk work |
| Retries | Automatic; past 20, “use a smaller upload file and try again” |
| Results kept | 7 days |
| API calls | Every HTTP request is one call from the org’s shared allocation |
The query side has its own list, in module 5: 10,000 query jobs and 1 TB of results a day.
That last row of the table is the one people forget. The dock isn’t free: each call to open, upload, close, check and download counts against the same allocation every other integration spends. It’s just very few calls for a lot of records. In class I put it this way: for 10 or 20 API calls, you can load a million records, against 5,000 at the teller window.
The Match: Steve Trucking at Year Three
Section titled “The Match: Steve Trucking at Year Three”So let’s make the match. Here’s module 1’s picture again, every feed in its place, with one more column: year three. At my baseline of 30% a year, that’s about 2.2 times today (The Number of Zeros).
Module 1’s feeds at year three: count again, and change the door only where the count crosses the line.
Three of the four are easy:
- Keystone’s nightly status file grows from about 8,800 rows to about 19,400. One query job a night, a few result sets. The dock’s limits are thousands of times bigger.
- The weekly export to Snowflake grows from about 23,000 deliveries to about 50,600. Still one query job a week.
- The RouteDesk migration happens once. There’s no year three for it.
The interesting one is Redrock. Today it sends one set per distribution center each evening: about 370 deliveries, about 810 at year three. That’s still under the line, and it still fits the teller window easily: a Composite call can carry five sObject Collections of 200 records each, so 810 records is one call (Many in One Call). And Redrock gets an answer for every delivery, right away.
But what if Redrock decides to send one set for all three centers? That’s about 1,100 today and about 2,400 at year three: past Salesforce’s line. At that volume, I’d move Redrock to the loading dock. The integration layer collects the manifests and the revisions as they come in, and loads them as a bulk job every 30 minutes. Why not every 15? A bulk job is asynchronous: it waits in Salesforce’s queue, and on a busy evening one run might not be finished before the next one starts. Thirty minutes gives each job room to finish, and the program still checks that the last run is complete and accounted for before it starts the next (module 6). The price is the answer: Redrock hears about each delivery within the half hour, from the results files, instead of straight away. That’s the trade to agree with Redrock before you switch.
The allocation passes too. In chapter 6’s “what if”, the org has 300,000 calls in 24 hours, shared by everyone (Constraints). Steve Trucking’s bulk jobs spend a few dozen of them a night. The calls to watch are the synchronous ones, like FleetLink’s, not the trucks.
Where the Line Moves
Section titled “Where the Line Moves”Two more things move the line, in opposite directions.
Small and often isn’t bulk. A feed that sends 300 records every 15 minutes adds up to almost 29,000 a day, past the line. But one trip is still 300 records, and that’s the count that matters. Sent as a bulk job, each trip is an open, an upload, a close, the checks and the downloads, with a job that might still be running when the next one starts. That’s the teller window’s work. The dock is for big trips, not many small ones.
Big and once can still be bulk, even when it’s quick. A one-time fix of 5,000 records, run by a person on a quiet afternoon, could go either way. If you want the results file and the count check from module 6, take the dock.
The Tools That Drive the Truck
Section titled “The Tools That Drive the Truck”Most bulk work isn’t hand-written calls. It’s a data loader, an ETL tool, an integration platform, a vendor’s connector. In class I called the bulk API the main gateway for getting data in and out, because they all sit on it.
But don’t assume the tool picked the dock. Salesforce’s own Data Loader says it plainly: “By default, Data Loader uses the SOAP-based API to process records.” You switch on Use Bulk API 2.0 in its settings to get the dock (Enable Bulk API 2.0 (opens in a new tab), Data Loader Guide, checked 2026-11-01). The same settings page has a caution worth repeating. With Bulk API 2.0 on, Data Loader can hard delete, and “hard-deleted records are immediately deleted and can’t be recovered from the Recycle Bin.” That’s why the “Bulk API Hard Delete” permission is off by default, and why I’d keep it off for integration users.
So for every tool on the landscape, ask three questions: which door does it use for each job, which version of the dock, and how does it sign in (module 3).
Bulk Is a Pattern, Not One Product
Section titled “Bulk Is a Pattern, Not One Product”One more thing to check before you quote a number: which dock you’re actually at. “Bulk” is a pattern, a job of CSV files, and it shows up in more than one Salesforce API, with very different limits.
Bulk API 2.0 itself now loads more than records in your org’s objects. Its job reference lists marketing objects (insert, upsert, and a refresh that’s for marketing objects only) and a consent import, each with its own operations (Create a Job (opens in a new tab), checked 2026-11-01).
And Data 360 has a bulk door of its own, the Ingestion API, for data going into Data 360 rather than into your org’s objects. Same idea, different numbers (Get Started with Ingestion API (opens in a new tab), Data 360 Ingestion API Reference, checked 2026-11-01):
| Bulk API 2.0 (your org’s objects) | Data 360 Ingestion API, bulk | |
|---|---|---|
| One file | 150 MB, encoded | 150 MB, up to 100 files a job |
| How many jobs | Limited by records: 150 million a day | 20 jobs an hour, 5 at one time |
| Open jobs | Close within 24 hours | Deleted after 7 days |
| When it’s ready to read | When the job’s complete | At least 30 seconds after, while caches refresh |
Data 360 also has a streaming door for small, frequent data: 200 KB a request, up to 250 requests a second across its endpoints. Don’t mix these numbers with Bulk API 2.0’s. Chapter 20, Data 360 and the Connected Enterprise (coming soon), takes Data 360’s doors in full.
Test knowledge. Know Salesforce’s line: more than about 2,000 records is a good candidate for Bulk API 2.0; fewer should use “bulkified” synchronous calls such as Composite. Know the main Bulk API 2.0 ingest limits: 150 million records in 24 hours, 150 MB per upload (about 100 MB before encoding), 15,000 batches a day shared with Bulk API 1.0, results kept 7 days, and that bulk calls still count against the org’s API allocation. Know that Data Loader uses the SOAP API unless Bulk API is switched on, and that hard delete skips the Recycle Bin and needs the “Bulk API Hard Delete” permission.
What an Architect Decides Here
Section titled “What an Architect Decides Here”- The door for each feed, from two questions: how many records in one trip, and whether the sender needs the answer now.
- The match at year three. Today’s count and year three’s against the dock’s limits, and against the allocation every integration shares. Write down the release you checked.
- The feeds near the line, and the change that would push them over: a partner batching differently, a new market, a backfill.
- The tools’ settings. Which tools use which door and which version, and who holds the hard-delete permission. For integration users, nobody.
- Which bulk door. Your org’s objects, marketing objects, or Data 360, and the limits that go with it.
Explore
Section titled “Explore”- Bulk API and Bulk API 2.0 Limits and Allocations (opens in a new tab): Salesforce Developer Limits and Allocations Quick Reference, the 2,000 line and the dock’s limits (checked 2026-11-01)
- Limits (opens in a new tab): Bulk API 2.0 and Bulk API Developer Guide, Winter ‘27, the planning questions and the CPU limit (checked 2026-11-01)
- Enable Bulk API 2.0 (opens in a new tab): Data Loader Guide (checked 2026-11-01)
- Get Started with Ingestion API (opens in a new tab): Data 360 Ingestion API Reference, its bulk and streaming limits (checked 2026-11-01)
- Constraints: chapter 6, the org’s shared allocation and the match this module makes
- Salesforce Developers, Bulk API 2.0 and Bulk API Developer Guide, Create a Job (opens in a new tab), checked 2026-11-07.