大数据量加载

另有 19 种语言版本

English · Español · 日本語 · Português (Brasil) · Français · Deutsch · Italiano · 한국어 · Português (Portugal) · 繁體中文 · Bahasa Indonesia · Türkçe · Polski · Tiếng Việt · ไทย · Українська · Español (México) · Français (Canada) · Deutsch (Schweiz)

In a greenhouse packing hall, three rows of tall plant trolleys, one all red flowers, one all yellow and one all white, wait by the open door. — 图片由 AI 生成

第 2 个模块，共 5 个，属于 Data Architect 课程第 10 章迁移与同步。每个模块都可以单独阅读。上一篇：迁移首先是一个质量项目 · 下一篇：加载期间的自动化

本模块回答的问题：“零的个数”如何改变你把数据加载到 Salesforce 的方式？

十条记录？我几乎可以手动输入。一千条？我需要一个工具。一百万条？那是一个架构问题。操作本身没变（创建或更新记录），但**“零的个数”**改变了我做它的方式。我们在建模、查询、共享和集成中都看到过这一点，迁移也一样。在大数据量下，我关心的是：如何移动数据，如何把它拆分成工作，按什么顺序移动，在我加载时 Salesforce 还在做什么，出了故障时如何恢复，以及能否把整个过程再跑一遍。

Start With the Picture

Same landscape, same amber arrow as module 1: RouteDesk’s open work coming into Salesforce the weekend RouteDesk retires. ⟲

点击放大

Module 1 decided what moves on this arrow. This module is about how it moves. ⟲

Module 1 decided what comes in: the deliveries not yet made and the ones still inside the 60-day claim window, about 196,400 deliveries. Each one becomes a Work Order, with its line items (about 1.6 per delivery, so about 314,000) and its Service Appointment (about 196,400 more). That’s roughly 707,000 records, prepared in the preparation data store and ready to go. Not millions, but well past anything I’d send one record at a time. So, how do they get in? ⟲

银行后面的装卸平台

讲 Bulk API 时，我用银行来比喻。你拿着一叠存款走进去，柜员收了 200 张，就让你回到队尾重新排队。如果有一百万条记录，那就是排 5,000 次队，5,000 次 API 调用。与此同时，一辆卡车开到银行侧面，卸下大量存款，二十分钟后就开走了，而你还在排队。那就是装卸平台，它的名字叫 Bulk API。

The 200 is still real. A REST sObject Collections request takes “up to 200 records” (Create Records Using sObject Collections （在新标签页中打开）, REST API Developer Guide), and a SOAP create() call the same: “If a create request exceeds 200 records, then the entire operation fails” (create() （在新标签页中打开）, SOAP API Developer Guide; both checked 2026-10-27). Every 200 more records is another call. RouteDesk’s 707,000 records would be over 3,500 trips to the teller. ⟲

点击放大

Same records, different door. ⟲

装卸平台有五个步骤：

**开启作业。**按下红色的蜂鸣器，拿到一张票：作业 ID。
**上传。**平台打开，你凭这张票装载你的 CSV 文件，一个文件或一百个都行。
**关闭作业。**在你按下那个蓝色大按钮之前，Salesforce 不知道你已经装完了：上传完成。现在它开始处理。
**“到了没有？”**就像我坐在后座的孩子们：你拿着作业 ID 问它完成了没有。完成了吗？完成了吗？
**取回结果。**下载成功的记录（带着它们新的 Salesforce ID）或失败的记录（带着它们的错误）。

点击放大

Submit, wait, collect. ⟲

一百万条记录只需要几次调用，而不是几千次。但注意变化在哪里：工作是异步的。你不会逐条记录得到答复；你提交、等待、收取。所以调用方需要知道提交了什么、什么还在处理、什么已完成、什么失败了、什么已经回来了。这不是 API 的细节。这是迁移的控制。

两个装卸平台：Bulk API 2.0 和 Bulk API

有两个版本，用的是哪一个很重要。

Bulk API 2.0 is the one to start with. You upload the data for a job and Salesforce splits it into batches for you, retries for you and runs them in parallel. A job can take 150 MB, measured after Salesforce encodes it, so Salesforce advises uploading no more than 100 MB of CSV per job. ⟲
Bulk API (1.0) makes you create the batches yourself, up to 10,000 records each. It’s older, but it still has two levers 2.0 doesn’t give you: serial mode for loads, and PK chunking for very large exports (module 5). ⟲

Both share a rolling 24-hour allowance of 15,000 batches, and either can load up to 150 million records in 24 hours. Job results are kept for seven days after the job completes (Bulk API and Bulk API 2.0 Limits and Allocations （在新标签页中打开）, checked 2026-10-27). Salesforce’s line between the doors: “Any data operation that includes more than 2,000 records is a good candidate for Bulk API 2.0”; below that, “bulkified” synchronous calls are fine (Loading Data from the API （在新标签页中打开）, Best Practices for Deployments with Large Data Volumes, Winter ‘27, checked 2026-10-27). Those are today’s numbers: check them before you plan around them. ⟲

单条、成组和批量，是我思考工作形状的方式：一次用户操作带来的单条更新，一个业务流程带来的几百条相关记录，一次迁移中的几百万条历史记录。这些不是产品名称。在选择工具之前，先问问工作的形状是什么。

Test knowledge. Know when the Bulk API is the answer: more than about 2,000 records, asynchronous, a job you submit and collect. Know the two versions apart: Bulk API 2.0 splits the batches for you and runs them in parallel; Bulk API (1.0) has you build batches of up to 10,000 records, and only it offers serial mode and PK chunking. Both share 15,000 batches in a rolling 24 hours. ⟲

工具消除不了架构

“我们会用 Bulk API。”好。这并没有告诉我哪些对象先加载，用插入还是 upsert，外部键是什么，关系如何重建，部分记录失败时会怎样，作业能否并行运行，并行的工作会不会争抢同样的父记录，会执行什么自动化，以及我们如何对账和重启。Bulk API 解决的是运输问题。它解决不了迁移。

The order comes from the model, and module 1 drew it for RouteDesk: reference data first, then the Work Orders, then their line items and appointments, then the claim Cases that point at them (Preserve the Source Identity, and Load by Keys). Each wave is its own job, and the next one doesn’t start until the last one’s results are in. Junction objects depend on two parents, self-referencing hierarchies may need several passes, and circular relationships may need records first and relationships second. Don’t discover these on cutover weekend. ⟲

有时你先加载记录，再加载关系。一家母公司、一家子公司和一家区域子公司都在迁移中创建。与其在一轮里解析所有关系，不如先带着稳定的源键加载这些客户，再跑第二轮来设置层级。同样的思路也适用于自引用查找和循环依赖。目标不是一轮完成。而是一个确定的、可重复的结果。

插入、更新还是 upsert

Insert says create this as new. Update says I already know the Salesforce record I’m changing. Upsert says use this key to find out whether the record exists; update it if it does, create it if it doesn’t. With stable External IDs, upsert makes rehearsals far easier: load, find a transformation problem, fix it, rerun, without the run assuming everything is a first-time insert. Keys are architecture, not just integration plumbing. RouteDesk’s Work Orders upsert on the partner key, the same field the integration uses, so a manifest Redrock resends on cutover weekend updates the migrated record instead of duplicating it. ⟲

Upsert has a price, though. Salesforce’s own Large Data Volumes guide ranks the operations: “insert() is fastest, update() is next, and upsert() is next after that. If possible, also break upsert() into two operations: create() and update()” (Loading Data from the API （在新标签页中打开）, checked 2026-10-27). How do you know which records are new and which already exist? The preparation data store knows, because it pulled down what Salesforce already has and keeps the IDs that came back (module 1). So the 190,000 delivered Work Orders that nothing in Salesforce has touched can go in as a plain insert, and only the open ones the integration may already have created need the upsert. ⟲

Two more lines from the same guide. Load clean data: “Errors in batches trigger single-row processing for that batch, and that processing heavily impacts performance.” That’s module 1 paying off at the loading dock. And when you update, “send only fields that have changed.” A reload that resends every field of 196,400 Work Orders does work nobody asked for. ⟲

但 upsert 不是魔法。如果键的策略错了，操作就是错的。同一个客户在源数据里有两条记录、带着不同的键，对 Salesforce 来说就是两条记录；一个能识别出多个目标的源键，说明我们误解了基数。模块 1 的质量工作依然重要；一个 API 功能取代不了它。

Test knowledge. Insert is the fastest operation, then update, then upsert; Salesforce suggests splitting an upsert into creates and updates where you can. Upsert needs an External ID, and it’s what makes reruns safe. Load clean data, because errors in a Bulk API batch push that batch into slow single-row processing, and send only the fields that changed. ⟲

并行更快，直到它不再更快

并行处理能提高吞吐量。但如果加载中的许多记录指向同一个父记录，事务就会争抢那个父记录上的锁，我们为了加快速度所做的事，反而造成了锁争用、错误和重试。这又是第 7 章的热点：如果一百万个子记录汇聚到几个父记录上，数据的形状对加载就很重要。当两个批次各自等待对方持有的锁时，你就得到了死锁，就像两辆车卡在一个路口。

RouteDesk is exactly that shape. Almost 196,400 Work Orders, and nearly every one points at one of three Retail Partner Accounts. Load them in parallel in the order RouteDesk exported them, and every batch holds Work Orders for all three partners. ⟲

点击放大

In export order, every batch needs every parent. One locks, the others wait. ⟲

Two levers. First, sort the files by parent, so the children of one parent travel together in the same batch instead of being spread across batches that collide. Salesforce gives the same advice: “When changing child records, group them by parent—group records by the field ParentId in the same batch to minimize locking conflicts” (Loading Data from the API （在新标签页中打开）, checked 2026-10-27). ⟲

点击放大

Sorted by partner, each batch works on its own parent. ⟲

Second, on Bulk API 1.0, serial mode processes one batch at a time: slower, but it removes the contention. I reach for serial mode when the data is complex and we’re running into collisions and timeouts. Salesforce’s own guide is more cautious: serial mode “is used only when previously submitted asynchronous jobs have accidentally produced contention because of locks. Use only when advised by Salesforce” (Limits （在新标签页中打开）, Bulk API 2.0 and Bulk API Developer Guide, checked 2026-10-27). So sort first, and treat serial as the fallback. And it’s only on the older dock: Bulk API 2.0’s job reference says “Currently only parallel mode is supported,” and if other modes ever arrive, “the mode will be chosen automatically by the API and will not be user configurable” (Create a Job （在新标签页中打开）, Bulk API 2.0, checked 2026-10-27). A load that needs serial processing is a reason to use Bulk API 1.0. ⟲

Serial mode is a setting on the job, chosen when you create it. On Bulk API 1.0, RouteDesk’s Work Orders would be: ⟲

POST /services/async/68.0/job
{
  "operation": "upsert",
  "object": "WorkOrder",
  "externalIdFieldName": "Partner_Delivery_Key__c",
  "contentType": "CSV",
  "concurrencyMode": "Serial"
}

Leave concurrencyMode out and you get Parallel, the default. Salesforce’s job reference says when to change it: “If parallel mode results in too many failed records or failed jobs, submit the job with Serial concurrency mode,” and adds that it “can significantly increase the processing time” (JobInfo （在新标签页中打开）, Bulk API 2.0 and Bulk API Developer Guide, checked 2026-10-31). ⟲

And if three parents are too few for any sorting to help, look at the model. Checkpoint 1 already offered a way to spread them: link Work Orders to a child Account per distribution center, so nine parents share the load instead of three. That’s a design decision, made before the migration, not a loader setting. ⟲

我不会一开始就说“永远并行”或“永远串行”。我看关系的形状、热门的父记录、会触发的自动化、还有什么会加锁，以及演练中发生了什么。然后再选择。先求正确，再求速度。

Test knowledge. Parallel loads lock parent records. When many children share a few parents, expect “unable to lock row” errors: sort the load file by parent so each batch touches as few parents as possible. Serial mode (Bulk API 1.0 only) removes the contention at the cost of speed; Salesforce reserves it for lock contention the sorting didn’t solve. ⟲

批次大小是一个调优决定

人们想要一个神奇的数字。我没有。合适的大小取决于对象、记录的复杂度、自动化、关系、锁定、负载和机制。太大，失败的代价就高，事务要做的工作也太多；太小，开销就占了大头，运行时间比应该的长得多（在 1.0 上还会耗掉你的批次额度）。从工具支持的限制范围内开始，然后用有代表性的数据来测量。

And the rehearsal data needs the same shape as production. Twenty million records in production, ten thousand in the test, and everything flies: what did we prove? Not much about scale. The test doesn’t need every record, but it must expose the architecture: heavy concentration on a few parents, millions of children, large text fields, complex relationships, historical skew, automation triggered by particular values. The Number of Zeros matters, and so does the shape behind the zeros. For RouteDesk that means a rehearsal set where a third of the Work Orders still point at Redrock. ⟲

加载用户，以及记录归谁所有

For a substantial migration I want a dedicated migration identity, not someone’s personal admin account: for traceability, controlled permissions, automation decisions (module 3), monitoring and troubleshooting. A million records created by the migration user during cutover tells me something. A million created by Daniel Reyes because Daniel ran Data Loader on Saturday night mixes human identity with system activity. Give it the permissions the job needs, and don’t let it become a permanent mystery superuser nobody dares disable. ⟲

Then don’t confuse who loaded a record with who should own it. Created By is provenance; Owner is architecture, with sharing and business consequences. Set business ownership deliberately. Sometimes there’s no good answer in the source: the record’s owner left years ago, or never existed. It can be hard to assign records to active users, and I’ve created a dedicated “data load” user to own them. That’s sound practice with one condition from chapter 7: a user who owns more than 10,000 records of an object is ownership skew, so that owner should have no role, or a role of its own at the top of the hierarchy (Ownership Data Skew （在新标签页中打开）), and the records should have a path to real owners. Who owns the records, and where that owner sits in the role hierarchy, is an important decision the team makes for any bulk import, not something the loader defaults. For RouteDesk’s Work Orders, chapter 7’s question decides it: does ownership drive who sees a delivery? At Steve Trucking it doesn’t (partners see their own deliveries through the partner on the record), so one owner with no role is fine. ⟲

工具：按可重复性来选

There are many ways to load data, and the tool matters less than how you use it. Salesforce’s Data Loader has been tried and true, but it’s aged: a desktop tool that works CSV by CSV. Its command-line version can be scripted and pointed at your preparation data store, which is how I’ve run repeated loads. Some tools connect a SQL database straight to Salesforce, copying data down and sending changes back with SQL. ETL and integration platforms do the same job at enterprise scale. And with the APIs you can drive the Bulk API directly: the Salesforce CLI’s sf data upsert bulk loads a CSV through Bulk API 2.0 with an External ID from one command (data upsert bulk （在新标签页中打开）, Salesforce CLI Command Reference, checked 2026-10-27). You can even have AI write the small scripts that run each complex step. ⟲

无论用什么工具，这是我最强烈的偏好：**我不希望生产迁移是我们第一次执行这次迁移，**也不希望每次演练都是一场手工打造的冒险。如果第 17 步写着“打开这个 CSV，按 G 列排序，删除空行，把这个公式往下拖，然后用这些设置运行 Data Loader”，我就会紧张。每一个手动步骤都可能带来错误的文件、漏掉的行、过时的版本、错误的排序、忘了的筛选条件、与演练不同的设置。有些也许无法避免：把它们写进文档并测试。但目标是可重复性，所以提取、转换、验证、加载、收集结果和对账，每一次都应该以同样的方式运行。

如果你按计划运行加载，就不要把频率安排得比它能完成的速度还快。一个每小时运行、有时却要九十分钟的作业，会叠在自己上面启动；在下一次运行开始之前，先检查上一次是否已经完成。

Keep Every Result, and Bring the Keys Back

对每一次加载，保留输入版本、转换后的加载文件、作业 ID、成功和失败的结果及其错误、耗时、记录数和对账输出。一次大型迁移之后，总会有人问“这条记录怎么了？”，我要的是证据，而不是“我觉得那是在周六上午的第三个 CSV 里”。批量结果只保留七天，所以要边做边下载。以后再给这些工件定生命周期；在你仍然需要证明这次迁移的时候，不要把它们扔掉。

Then bring the keys back. The success results carry each record’s new Salesforce ID next to the key you sent. Write them back into the preparation data store, on the row that produced the record: that’s the last frame of module 1’s three sets of keys. On the Zendesk migration I described in module 1, that’s what made everything else possible. With the Salesforce IDs back on every row, I could load in phases (accounts and contacts first, then the cases, then the comments and files that hang off them), and each phase found its parents by the IDs the last one returned. I could do a partial load: just the rows that failed, or just one group. And I could reload: fix a transformation, regenerate the affected rows, and send only those, knowing exactly which Salesforce records they’d update. Without the keys back, every rerun is a guess. ⟲

失败的多少也是一个“零的个数”的问题。100 条里有 20 条失败很糟糕；2,000 万条里有 20 条失败依然需要调查，但它是一个不同的信号。看比率，而不只是数量，并对失败进行分类：验证、缺少父记录、重复的键、锁定、数据类型、自动化、权限。分好类之后，它们向你展示的是架构问题，而不是一个巨大的错误文件。

而且性能不只是每秒处理多少条记录。“我们把吞吐量翻了一番。”什么变了？锁定、失败或重试增加了吗？下游的自动化是不是跟不上了，共享重新计算是不是积压了，另一个集成是不是被淹没了？整个切换真的变短了吗？迁移不是在真空中运行的。

Test knowledge. The Bulk API’s success results return each record’s new Salesforce ID with the data you sent, and its failure results return each error; both are kept for seven days. Capture them, write the IDs back to the source or staging data, and you can load in phases, retry only the failures, and reconcile what landed. ⟲

测试整个流程

先加载客户，再加载联系人，再加载订单，每一步都单独测试过，这不叫测试过的迁移。我要的是一次完整的演练，从头到尾，用同样的顺序、转换、身份、自动化策略和对账，在有代表性的数据量下进行。依赖关系会在步骤之间显现：十分钟的延迟让下一次加载在错误的时间开始，一个意料之外的失败率花掉几个小时来修复，一次共享重新计算打乱了时间表。完整的运行告诉你迁移是否可行，而不只是每条命令是否可行。

If the rehearsal includes a performance or scale test, put Salesforce’s lead time in the plan. “All tests must be pre-approved. Unapproved testing is subject to throttling or blocking,” and for tests at 120 requests a second or below, the request goes in “at least one week before the testing date” (Salesforce Performance and Scale Test FAQs （在新标签页中打开）, Salesforce Help, checked 2026-10-27). A rehearsal that gets throttled proves nothing about cutover weekend. ⟲

AI 可以写脚本。运行仍然由你负责。

AI is very good at generating transformation scripts, load orchestration, error parsers, reconciliation queries and retry logic. Use it. But generated code can be wrong at enormous speed: a wrong transformation on ten records is ten fixes; on ten million, the Number of Zeros applies to the mistake too. The way I use it is step by step: clean, transform and migrate steps, with every generated command and action tracked. That’s deterministic, auditable automation, not unattended judgment. Test it at realistic scale before you trust it with production data. ⟲

**考试之外：AI 改变了什么。**AI 降低了构建迁移工具的工作量，也让快速生成和运行批量操作变得更容易。操作规模越大，就越需要确定性的转换、被跟踪的步骤、演练、控制和对账。

架构师在这里决定什么

哪种加载机制和 API 版本适合工作的数据量和形状，而不是把迁移当作事务性流量。
由依赖关系驱动的对象和关系加载顺序，包括记录和关系什么时候需要分开处理。
Which operation each set of records gets (insert, update or upsert), based on what the preparation data store knows already exists. ⟲
Where parallel processing is safe, and where concentration, locking or automation justify sorting by parent, a model change, or serial processing. ⟲
哪些稳定的键、迁移身份、记录所有权和可重复的工具，能让演练和生产加载都是确定性的。
How results and new IDs come back, so loads can run in phases, retry only failures, and reload safely. ⟲
在生产规模下信任这个流程之前，需要什么证据、失败分类和完整的流程演练。

装卸平台依然需要一个计划

The loading dock moves enormous amounts of data. It doesn’t decide which truck unloads first, where the pallets go, which shipment belongs to which customer, what happens to a damaged pallet, whether two forklifts are using the same aisle, or whether everything that should have arrived did. That’s architecture. Know the Number of Zeros and the shape behind them. Load in dependency order, use stable keys, control concurrency, bring the keys back, make the process repeatable, keep the evidence, and rehearse the whole thing. Then use the loading dock as fast as the architecture safely allows. ⟲

延伸探索

Salesforce, Bulk API and Bulk API 2.0 Limits and Allocations （在新标签页中打开）, Developer Limits and Allocations Quick Reference, checked 2026-11-01. ⟲
Salesforce, General Guidelines for Data Loads （在新标签页中打开） and Limits （在新标签页中打开）, Bulk API 2.0 and Bulk API Developer Guide, checked 2026-11-01. ⟲
Salesforce, Best Practices for Deployments with Large Data Volumes, Loading Data from the API （在新标签页中打开）, Winter ‘27, checked 2026-11-01. ⟲
Salesforce, REST API Developer Guide, Create Records Using sObject Collections （在新标签页中打开）, checked 2026-11-01. ⟲
Salesforce, SOAP API Developer Guide, create() （在新标签页中打开）, and Bulk API 2.0, Create a Job （在新标签页中打开）, checked 2026-11-01. ⟲
Salesforce, Bulk API 2.0 and Bulk API Developer Guide, JobInfo （在新标签页中打开） (the Bulk API 1.0 job fields, concurrencyMode), API version 68.0 (Winter ‘27), checked 2026-11-01. ⟲
Salesforce Help, Salesforce Performance and Scale Test FAQs （在新标签页中打开）, checked 2026-11-01. ⟲
Integration Architect 课程：Bulk (coming soon) 深入讲解了 Bulk API。
本课程中：为规模而设计（热点和所有权倾斜）。
Salesforce Developers, Designing Record Access for Enterprise Scale, Ownership Data Skew （在新标签页中打开）, checked 2026-11-01. ⟲
Salesforce Developers, Salesforce CLI Command Reference, data upsert bulk （在新标签页中打开）, checked 2026-11-01. ⟲