Bulk Query

Module 5 of 8 in the chapter Moving Data at Scale, chapter 10 of the Integration Architect course. Each module stands on its own. Previous: Parallel and Locking · Next: Did It All Arrive?

The question this module answers: How does a program take a million records out of Salesforce, and what’s different from putting them in?

A Million Records Out

There are two kinds of bulk jobs. Ingest jobs put data in: insert, update, upsert, delete. Query jobs take data out. In class I ask: if you ran a query through the REST API to pull a million records out of Salesforce, what would happen?

The REST API won’t hand you a million records in one answer. It returns “up to 2,000 records… at a time in a synchronous request,” with a nextRecordsUrl for the next set (Query (opens in a new tab), REST API Developer Guide, API 68.0, checked 2026-11-01). So a million records is about 500 calls, one after another, each one taken from the org’s API allowance, and the whole time the program is waiting at the teller window. That’s not the design you want. Salesforce draws the line for you: “When using a query that can return more than one million results, consider using the query capability of the Bulk API 2.0” (Extracting Data from the API (opens in a new tab), Best Practices for Deployments with Large Data Volumes, checked 2026-11-01).

So the loading dock works in both directions. Sometimes the truck is unloading, and sometimes it’s being loaded.

Three Calls and a Download

A query job is shorter than an ingest job. There’s nothing to upload, so there’s no step two and no buzzer to ring. You open the job with the SOQL, and Salesforce is ready to run it straight away. The work moves to the other end: getting the results back.

Click to enlarge

Compare it with the ingest job in module 2: no upload, no “upload complete.” The work is in the download.

1. Open the query job. A POST to /services/data/v68.0/jobs/query, with the SOQL:

POST /services/data/v68.0/jobs/query
Authorization: Bearer <access token>
Content-Type: application/json

{
  "operation": "query",
  "query": "SELECT Id, Partner_Delivery_Key__c, Status, SystemModstamp FROM WorkOrder WHERE SystemModstamp = YESTERDAY"
}

The answer has the job ID, and the job’s state is already UploadComplete: for a query, that means “the job is ready to be processed” (Job States (opens in a new tab), Bulk API 2.0 and Bulk API Developer Guide, Winter ‘27, checked 2026-11-01). The operation is query for the records you’d see in Salesforce, or queryAll to include records “deleted because of a merge or delete” and archived Tasks and Events (Create a Query Job (opens in a new tab), same guide, checked 2026-11-01). queryAll matters when you’re keeping a copy, and that’s module 8.

2. Are we there yet? A GET on /jobs/query/<job ID> until the state is JobComplete (or Failed, or Aborted). This is the one place the 2026 answer differs from the ingest side. For query jobs, Salesforce can tell you: the BulkApi2JobEvent platform event announces the job’s status and gives you URLs for results that are ready, so you can start downloading before the whole job is done. It’s still a Beta, it’s turned on in Setup, and it goes to programs listening through the Pub/Sub API or the older Streaming API (module 2 has the details). Until you’ve decided to build on a Beta, ask.

3. Download the results, set by set. A GET on /jobs/query/<job ID>/results returns the first set as CSV. Two response headers tell you where you are: Sforce-NumberOfRecords, the records in this set, and Sforce-Locator, the key to the next set. Ask again with ?locator= and that value, and keep going until the locator comes back as the string null. Salesforce adds two rules worth putting in the code: “use only the value from the Sforce-Locator header. Don’t try to guess what it is,” and use the same API version you created the job with, or the call fails with a 409 (Get Results for a Query Job (opens in a new tab), same guide, checked 2026-11-01).

The maxRecords parameter sets the size of each set. For very large results, the guide’s advice is to set it, because “you may experience a timeout before receiving all the data.” Smaller sets, more of them, each one safe to download. And a set can be downloaded again: the locator “does not change,” so a program that fails halfway can pick up where it stopped instead of starting over.

Or five sets at once. From API 58.0, a GET on /jobs/query/<job ID>/resultPages “returns up to five URIs in one response.” Each is a results URL with its own locator, and your program can fetch them at the same time, then ask for the next five until done is true (Get Parallel Results for a Query Job (opens in a new tab), same guide, checked 2026-11-01). On a big export, that’s the difference between one truck being unloaded through one door and five doors open.

The results come back compressed, and they wait for you for seven days. After that, the job and its results are deleted.

The SOQL It Won’t Run

A bulk query is SOQL, but not every SOQL. In class I warned that it looks simple until it doesn’t. Salesforce’s current list of what Bulk API 2.0 doesn’t support (Understanding Bulk API 2.0 Query (opens in a new tab), same guide, checked 2026-11-01):

GROUP BY, OFFSET or TYPEOF
aggregate functions, such as COUNT()
date functions in GROUP BY (they’re fine in WHERE)
compound address and geolocation fields, and FIELDS(): ask for the parts instead
parent-to-child subqueries. Child-to-parent is fine: Account.Name from a Work Order works; a list of Work Orders inside each Account doesn’t.

A bulk query is for moving rows, not for answering a question. If the other system needs totals, send it the rows and let it add them up, or build the summary somewhere designed for it.

Two more clauses are allowed but cost you: ORDER BY and LIMIT. They switch off the automatic chunking in the next section, and the guide’s advice when a query times out is to “remove the ORDER BY or LIMIT clause before any subsequent troubleshooting.” Sort the file after it arrives, not in the query.

Chunking: Automatic Now, a Header on 1.0

Pulling a million records out of a 50-million-row table can be slow, because the database may be looking at all 50 million to find yours. The answer is to cut the table into ranges of record IDs and query each range on its own. On Bulk API 2.0, you don’t do anything: query jobs “automatically determine the best way to divide your query job into smaller chunks, helping to avoid failures or timeouts,” for standard objects that support it, custom objects, and their Sharing and History tables. The job’s isPkChunkingSupported field tells you whether yours does.

On Bulk API 1.0, it’s a header you set yourself, Sforce-Enable-PKChunking, with your own chunk size. That’s the one I said is on the test, and it still is. The Data Architect course draws it on Steve Trucking’s Work Orders:

Click to enlarge

From the Data Architect course: one big table becomes about 52 queries of 250,000 records.

What you need to know about the 1.0 header (PK Chunking (opens in a new tab), same guide, checked 2026-11-01):

When. Salesforce “recommends that you enable PK chunking when querying tables with more than 10 million records or when a bulk query consistently times out.”
The chunk size. The default is 100,000 records, the maximum 250,000.
What it costs. “Each chunk is processed as a separate batch that counts toward your daily batch limit, and you must download each batch’s results separately.” On 2.0, query jobs don’t use the 15,000 daily batches at all.
What switches it off. ORDER BY, LIMIT, and filtering on “any field with ‘id’ in the field name.”

So if you’re writing a new export today, it’s a 2.0 query job and the chunking is Salesforce’s problem. You’ll meet the header in older programs and on the exam.

Its Constraints

These are the numbers to hold your feeds against, from Salesforce’s limits page (Bulk API and Bulk API 2.0 Limits and Allocations (opens in a new tab), Salesforce Developer Limits and Allocations Quick Reference, checked 2026-11-01):

Bulk API 2.0 query	Limit
Query jobs in a rolling 24 hours	10,000 (`DailyBulkV2QueryJobs` in the REST `/limits` resource)
Query results generated in a rolling 24 hours	1 TB (`DailyBulkV2QueryFileStorageMB`)
Retries	Automatic; past 15, “apply a filter criteria and try again”
Time to retrieve the results	20 minutes
Results kept	7 days
Daily batches used	None: only ingest jobs use the 15,000

Both /limits values are readable by your program, so a nightly feed can check them before it starts, not after it fails. Module 7 puts these numbers next to Steve Trucking’s feeds and year three’s growth.

Steve Trucking’s Exports

Two of Steve Trucking’s feeds are bulk queries, and neither one comes near the limits today:

Keystone’s nightly status file. About 1,100 Keystone deliveries a day, about eight status updates each: about 8,800 rows a night, read out of Salesforce and written to a file for Keystone. Past the 2,000 line, so a query job: open it, wait, download a handful of sets.
The weekly export to Snowflake. About 23,000 deliveries a week and their status history. The same three calls, and a good place for the five-at-once download.

The table that will need chunking is the Work Orders themselves. At a million deliveries a year growing 30 percent, Work Orders pass 10 million in year six unless the archiving in the Data Architect course keeps the table smaller. On 2.0, the query job chunks that for you, as long as nobody adds an ORDER BY.

Your Lab Bench

Try it in your Developer Edition org, in Postman, signed in the same way as chapter 8’s lab bench. Make a few made-up accounts first, then:

Open a query job with SELECT Id, Name, SystemModstamp FROM Account. Notice the state in the answer: it’s already UploadComplete.
Ask for its status until it’s JobComplete.
Download the results with ?maxRecords=2, so even a small org gives you more than one set. Look at the Sforce-Locator header, ask again with ?locator= and that value, and keep going until it says null. Add up Sforce-NumberOfRecords as you go: that’s module 6’s count.
Ask for resultPages on the same job, and see the result URLs it hands back.
Delete one of your made-up accounts, then run the same query again as queryAll. The deleted account comes back; with query, it doesn’t. That’s module 8’s deletes, in your hands.

Then break it. Open a job with SELECT COUNT() FROM Account, or with a subquery of Contacts inside each Account, and read what Salesforce tells you. Run the same COUNT() through the REST query resource instead, and it works: that’s the other door doing the counting.

Test knowledge. Know that bulk query jobs move large result sets out of Salesforce asynchronously, and that Salesforce suggests Bulk API 2.0 query for results over about a million records. Know what bulk SOQL won’t run: GROUP BY, OFFSET, TYPEOF, aggregates such as COUNT(), compound fields, parent-to-child subqueries. Know PK chunking: it splits a query on a large table into ranges of record IDs (default chunk 100,000, maximum 250,000), recommended past 10 million records or when a bulk query keeps timing out; it’s a header on Bulk API 1.0 and automatic on 2.0; ORDER BY and LIMIT switch it off. Know query from queryAll (which includes deleted and archived records).

What an Architect Decides Here

Which door each export uses. Count the records in one trip. A few thousand: REST query, set by set. Past about a million, or every night at volume: a Bulk API 2.0 query job.
The SOQL, written for bulk. No aggregates or subqueries, no ORDER BY or LIMIT on a big table. Totals and sorting happen in the receiving system.
How the results come down. Set by set with maxRecords sized for the client, or five at a time. And what the program does if it fails halfway: pick up at the last locator.
How the program knows the job is done. Asking, or the job event, and whether you’ll build on a Beta.
query or queryAll. Whether the other system needs to hear about deleted records (module 8).
The allowance. Whether the nightly and weekly exports, together, fit the daily query jobs and result storage, now and at year three.

Explore

Understanding Bulk API 2.0 Query (opens in a new tab), Get Results for a Query Job (opens in a new tab) and Get Parallel Results for a Query Job (opens in a new tab): Bulk API 2.0 and Bulk API Developer Guide, Winter ‘27 (checked 2026-11-01)
PK Chunking (opens in a new tab): the same guide, Bulk API 1.0’s header and the supported objects (checked 2026-11-01)
Extracting Data from the API (opens in a new tab): Best Practices for Deployments with Large Data Volumes (checked 2026-11-01)
Bulk API and Bulk API 2.0 Limits and Allocations (opens in a new tab): Salesforce Developer Limits and Allocations Quick Reference (checked 2026-11-01)
Getting Data Out: Keeping Another System in Step: the Data Architect course, the ways data comes out of Salesforce and PK chunking for a migration
Salesforce Developers, REST API Developer Guide, Query (opens in a new tab), checked 2026-11-07.
Salesforce Developers, Bulk API 2.0 and Bulk API Developer Guide, Job States (opens in a new tab), checked 2026-11-07.
Salesforce Developers, Bulk API 2.0 and Bulk API Developer Guide, Create a Query Job (opens in a new tab), checked 2026-11-07.