大數據量載入

另有 19 種語言版本

English · Español · 日本語 · Português (Brasil) · Français · Deutsch · Italiano · 한국어 · 简体中文 · Português (Portugal) · Bahasa Indonesia · Türkçe · Polski · Tiếng Việt · ไทย · Українська · Español (México) · Français (Canada) · Deutsch (Schweiz)

In a greenhouse packing hall, three rows of tall plant trolleys, one all red flowers, one all yellow and one all white, wait by the open door. — 圖片由 AI 生成

第 2 個模組，共 5 個，屬於 Data Architect 課程第 10 章遷移與同步。每個模組都可以單獨閱讀。上一篇：遷移首先是一個品質專案 · 下一篇：載入期間的自動化

本模組回答的問題：「零的個數」如何改變你把資料載入到 Salesforce 的方式？

十條記錄？我幾乎可以手動輸入。一千條？我需要一個工具。一百萬條？那是一個架構問題。操作本身沒變（建立或更新記錄），但**「零的個數」**改變了我做它的方式。我們在建模、查詢、共用和整合中都看到過這一點，遷移也一樣。在大數據量下，我關心的是：如何移動資料，如何把它拆分成工作，按什麼順序移動，在我載入時 Salesforce 還在做什麼，出了故障時如何恢復，以及能否把整個過程再跑一遍。

Start With the Picture

Same landscape, same amber arrow as module 1: RouteDesk’s open work coming into Salesforce the weekend RouteDesk retires. ⟲

點擊放大

Module 1 decided what moves on this arrow. This module is about how it moves. ⟲

Module 1 decided what comes in: the deliveries not yet made and the ones still inside the 60-day claim window, about 196,400 deliveries. Each one becomes a Work Order, with its line items (about 1.6 per delivery, so about 314,000) and its Service Appointment (about 196,400 more). That’s roughly 707,000 records, prepared in the preparation data store and ready to go. Not millions, but well past anything I’d send one record at a time. So, how do they get in? ⟲

銀行後面的裝卸平台

講 Bulk API 時，我用銀行來比喻。你拿著一疊存款走進去，櫃員收了 200 張，就讓你回到隊尾重新排隊。如果有一百萬條記錄，那就是排 5,000 次隊，5,000 次 API 呼叫。與此同時，一輛卡車開到銀行側面，卸下大量存款，二十分鐘後就開走了，而你還在排隊。那就是裝卸平台，它的名字叫 Bulk API。

The 200 is still real. A REST sObject Collections request takes “up to 200 records” (Create Records Using sObject Collections （在新標籤頁中開啟）, REST API Developer Guide), and a SOAP create() call the same: “If a create request exceeds 200 records, then the entire operation fails” (create() （在新標籤頁中開啟）, SOAP API Developer Guide; both checked 2026-10-27). Every 200 more records is another call. RouteDesk’s 707,000 records would be over 3,500 trips to the teller. ⟲

點擊放大

Same records, different door. ⟲

裝卸平台有五個步驟：

**開啟作業。**按下紅色的蜂鳴器，拿到一張票：作業 ID。
**上傳。**平台開啟，你憑這張票裝載你的 CSV 檔案，一個檔案或一百個都行。
**關閉作業。**在你按下那個藍色大按鈕之前，Salesforce 不知道你已經裝完了：上傳完成。現在它開始處理。
**「到了沒有？」**就像我坐在後座的孩子們：你拿著作業 ID 問它完成了沒有。完成了嗎？完成了嗎？
**取回結果。**下載成功的記錄（帶著它們新的 Salesforce ID）或失敗的記錄（帶著它們的錯誤）。

點擊放大

Submit, wait, collect. ⟲

一百萬條記錄只需要幾次呼叫，而不是幾千次。但注意變化在哪裡：工作是非同步的。你不會逐條記錄得到答覆；你提交、等待、收取。所以呼叫方需要知道提交了什麼、什麼還在處理、什麼已完成、什麼失敗了、什麼已經回來了。這不是 API 的細節。這是遷移的控制。

兩個裝卸平台：Bulk API 2.0 和 Bulk API

有兩個版本，用的是哪一個很重要。

Bulk API 2.0 is the one to start with. You upload the data for a job and Salesforce splits it into batches for you, retries for you and runs them in parallel. A job can take 150 MB, measured after Salesforce encodes it, so Salesforce advises uploading no more than 100 MB of CSV per job. ⟲
Bulk API (1.0) makes you create the batches yourself, up to 10,000 records each. It’s older, but it still has two levers 2.0 doesn’t give you: serial mode for loads, and PK chunking for very large exports (module 5). ⟲

Both share a rolling 24-hour allowance of 15,000 batches, and either can load up to 150 million records in 24 hours. Job results are kept for seven days after the job completes (Bulk API and Bulk API 2.0 Limits and Allocations （在新標籤頁中開啟）, checked 2026-10-27). Salesforce’s line between the doors: “Any data operation that includes more than 2,000 records is a good candidate for Bulk API 2.0”; below that, “bulkified” synchronous calls are fine (Loading Data from the API （在新標籤頁中開啟）, Best Practices for Deployments with Large Data Volumes, Winter ‘27, checked 2026-10-27). Those are today’s numbers: check them before you plan around them. ⟲

單條、成組和批次，是我思考工作形狀的方式：一次使用者操作帶來的單條更新，一個業務流程帶來的幾百條相關記錄，一次遷移中的幾百萬條歷史記錄。這些不是產品名稱。在選擇工具之前，先問問工作的形狀是什麼。

Test knowledge. Know when the Bulk API is the answer: more than about 2,000 records, asynchronous, a job you submit and collect. Know the two versions apart: Bulk API 2.0 splits the batches for you and runs them in parallel; Bulk API (1.0) has you build batches of up to 10,000 records, and only it offers serial mode and PK chunking. Both share 15,000 batches in a rolling 24 hours. ⟲

工具消除不了架構

「我們會用 Bulk API。」好。這並沒有告訴我哪些物件先載入，用插入還是 upsert，外部鍵是什麼，關係如何重建，部分記錄失敗時會怎樣，作業能否並行執行，並行的工作會不會爭搶同樣的父記錄，會執行什麼自動化，以及我們如何對賬和重啟。Bulk API 解決的是運輸問題。它解決不了遷移。

The order comes from the model, and module 1 drew it for RouteDesk: reference data first, then the Work Orders, then their line items and appointments, then the claim Cases that point at them (Preserve the Source Identity, and Load by Keys). Each wave is its own job, and the next one doesn’t start until the last one’s results are in. Junction objects depend on two parents, self-referencing hierarchies may need several passes, and circular relationships may need records first and relationships second. Don’t discover these on cutover weekend. ⟲

有時你先載入記錄，再載入關係。一家母公司、一家子公司和一家區域子公司都在遷移中建立。與其在一輪裡解析所有關係，不如先帶著穩定的源鍵載入這些客戶，再跑第二輪來設定層級。同樣的思路也適用於自引用查閱和迴圈依賴。目標不是一輪完成。而是一個確定的、可重複的結果。

插入、更新還是 upsert

Insert says create this as new. Update says I already know the Salesforce record I’m changing. Upsert says use this key to find out whether the record exists; update it if it does, create it if it doesn’t. With stable External IDs, upsert makes rehearsals far easier: load, find a transformation problem, fix it, rerun, without the run assuming everything is a first-time insert. Keys are architecture, not just integration plumbing. RouteDesk’s Work Orders upsert on the partner key, the same field the integration uses, so a manifest Redrock resends on cutover weekend updates the migrated record instead of duplicating it. ⟲

Upsert has a price, though. Salesforce’s own Large Data Volumes guide ranks the operations: “insert() is fastest, update() is next, and upsert() is next after that. If possible, also break upsert() into two operations: create() and update()” (Loading Data from the API （在新標籤頁中開啟）, checked 2026-10-27). How do you know which records are new and which already exist? The preparation data store knows, because it pulled down what Salesforce already has and keeps the IDs that came back (module 1). So the 190,000 delivered Work Orders that nothing in Salesforce has touched can go in as a plain insert, and only the open ones the integration may already have created need the upsert. ⟲

Two more lines from the same guide. Load clean data: “Errors in batches trigger single-row processing for that batch, and that processing heavily impacts performance.” That’s module 1 paying off at the loading dock. And when you update, “send only fields that have changed.” A reload that resends every field of 196,400 Work Orders does work nobody asked for. ⟲

但 upsert 不是魔法。如果鍵的策略錯了，操作就是錯的。同一個客戶在源資料裡有兩條記錄、帶著不同的鍵，對 Salesforce 來說就是兩條記錄；一個能識別出多個目標的源鍵，說明我們誤解了基數。模組 1 的品質工作依然重要；一個 API 功能取代不了它。

Test knowledge. Insert is the fastest operation, then update, then upsert; Salesforce suggests splitting an upsert into creates and updates where you can. Upsert needs an External ID, and it’s what makes reruns safe. Load clean data, because errors in a Bulk API batch push that batch into slow single-row processing, and send only the fields that changed. ⟲

並行更快，直到它不再更快

並行處理能提高吞吐量。但如果載入中的許多記錄指向同一個父記錄，事務就會爭搶那個父記錄上的鎖，我們為了加快速度所做的事，反而造成了鎖爭用、錯誤和重試。這又是第 7 章的熱點：如果一百萬個子記錄匯聚到幾個父記錄上，資料的形狀對載入就很重要。當兩個批次各自等待對方持有的鎖時，你就得到了死鎖，就像兩輛車卡在一個路口。

RouteDesk is exactly that shape. Almost 196,400 Work Orders, and nearly every one points at one of three Retail Partner Accounts. Load them in parallel in the order RouteDesk exported them, and every batch holds Work Orders for all three partners. ⟲

點擊放大

In export order, every batch needs every parent. One locks, the others wait. ⟲

Two levers. First, sort the files by parent, so the children of one parent travel together in the same batch instead of being spread across batches that collide. Salesforce gives the same advice: “When changing child records, group them by parent—group records by the field ParentId in the same batch to minimize locking conflicts” (Loading Data from the API （在新標籤頁中開啟）, checked 2026-10-27). ⟲

點擊放大

Sorted by partner, each batch works on its own parent. ⟲

Second, on Bulk API 1.0, serial mode processes one batch at a time: slower, but it removes the contention. I reach for serial mode when the data is complex and we’re running into collisions and timeouts. Salesforce’s own guide is more cautious: serial mode “is used only when previously submitted asynchronous jobs have accidentally produced contention because of locks. Use only when advised by Salesforce” (Limits （在新標籤頁中開啟）, Bulk API 2.0 and Bulk API Developer Guide, checked 2026-10-27). So sort first, and treat serial as the fallback. And it’s only on the older dock: Bulk API 2.0’s job reference says “Currently only parallel mode is supported,” and if other modes ever arrive, “the mode will be chosen automatically by the API and will not be user configurable” (Create a Job （在新標籤頁中開啟）, Bulk API 2.0, checked 2026-10-27). A load that needs serial processing is a reason to use Bulk API 1.0. ⟲

Serial mode is a setting on the job, chosen when you create it. On Bulk API 1.0, RouteDesk’s Work Orders would be: ⟲

POST /services/async/68.0/job
{
  "operation": "upsert",
  "object": "WorkOrder",
  "externalIdFieldName": "Partner_Delivery_Key__c",
  "contentType": "CSV",
  "concurrencyMode": "Serial"
}

Leave concurrencyMode out and you get Parallel, the default. Salesforce’s job reference says when to change it: “If parallel mode results in too many failed records or failed jobs, submit the job with Serial concurrency mode,” and adds that it “can significantly increase the processing time” (JobInfo （在新標籤頁中開啟）, Bulk API 2.0 and Bulk API Developer Guide, checked 2026-10-31). ⟲

And if three parents are too few for any sorting to help, look at the model. Checkpoint 1 already offered a way to spread them: link Work Orders to a child Account per distribution center, so nine parents share the load instead of three. That’s a design decision, made before the migration, not a loader setting. ⟲

我不會一開始就說「永遠並行」或「永遠序列」。我看關係的形狀、熱門的父記錄、會觸發的自動化、還有什麼會加鎖，以及演練中發生了什麼。然後再選擇。先求正確，再求速度。

Test knowledge. Parallel loads lock parent records. When many children share a few parents, expect “unable to lock row” errors: sort the load file by parent so each batch touches as few parents as possible. Serial mode (Bulk API 1.0 only) removes the contention at the cost of speed; Salesforce reserves it for lock contention the sorting didn’t solve. ⟲

批次大小是一個調優決定

人們想要一個神奇的數字。我沒有。合適的大小取決於物件、記錄的複雜度、自動化、關係、鎖定、負載和機制。太大，失敗的代價就高，事務要做的工作也太多；太小，開銷就佔了大頭，執行時間比應該的長得多（在 1.0 上還會耗掉你的批次額度）。從工具支援的限制範圍內開始，然後用有代表性的資料來測量。

And the rehearsal data needs the same shape as production. Twenty million records in production, ten thousand in the test, and everything flies: what did we prove? Not much about scale. The test doesn’t need every record, but it must expose the architecture: heavy concentration on a few parents, millions of children, large text fields, complex relationships, historical skew, automation triggered by particular values. The Number of Zeros matters, and so does the shape behind the zeros. For RouteDesk that means a rehearsal set where a third of the Work Orders still point at Redrock. ⟲

載入使用者，以及記錄歸誰所有

For a substantial migration I want a dedicated migration identity, not someone’s personal admin account: for traceability, controlled permissions, automation decisions (module 3), monitoring and troubleshooting. A million records created by the migration user during cutover tells me something. A million created by Daniel Reyes because Daniel ran Data Loader on Saturday night mixes human identity with system activity. Give it the permissions the job needs, and don’t let it become a permanent mystery superuser nobody dares disable. ⟲

Then don’t confuse who loaded a record with who should own it. Created By is provenance; Owner is architecture, with sharing and business consequences. Set business ownership deliberately. Sometimes there’s no good answer in the source: the record’s owner left years ago, or never existed. It can be hard to assign records to active users, and I’ve created a dedicated “data load” user to own them. That’s sound practice with one condition from chapter 7: a user who owns more than 10,000 records of an object is ownership skew, so that owner should have no role, or a role of its own at the top of the hierarchy (Ownership Data Skew （在新標籤頁中開啟）), and the records should have a path to real owners. Who owns the records, and where that owner sits in the role hierarchy, is an important decision the team makes for any bulk import, not something the loader defaults. For RouteDesk’s Work Orders, chapter 7’s question decides it: does ownership drive who sees a delivery? At Steve Trucking it doesn’t (partners see their own deliveries through the partner on the record), so one owner with no role is fine. ⟲

工具：按可重複性來選

There are many ways to load data, and the tool matters less than how you use it. Salesforce’s Data Loader has been tried and true, but it’s aged: a desktop tool that works CSV by CSV. Its command-line version can be scripted and pointed at your preparation data store, which is how I’ve run repeated loads. Some tools connect a SQL database straight to Salesforce, copying data down and sending changes back with SQL. ETL and integration platforms do the same job at enterprise scale. And with the APIs you can drive the Bulk API directly: the Salesforce CLI’s sf data upsert bulk loads a CSV through Bulk API 2.0 with an External ID from one command (data upsert bulk （在新標籤頁中開啟）, Salesforce CLI Command Reference, checked 2026-10-27). You can even have AI write the small scripts that run each complex step. ⟲

無論用什麼工具，這是我最強烈的偏好：**我不希望生產遷移是我們第一次執行這次遷移，**也不希望每次演練都是一場手工打造的冒險。如果第 17 步寫著「開啟這個 CSV，按 G 列排序，刪除空行，把這個公式往下拖，然後用這些設定執行 Data Loader」，我就會緊張。每一個手動步驟都可能帶來錯誤的檔案、漏掉的行、過時的版本、錯誤的排序、忘了的篩選條件、與演練不同的設定。有些也許無法避免：把它們寫進文件並測試。但目標是可重複性，所以提取、轉換、驗證、載入、收集結果和對賬，每一次都應該以同樣的方式執行。

如果你按計劃執行載入，就不要把頻率安排得比它能完成的速度還快。一個每小時執行、有時卻要九十分鐘的作業，會疊在自己上面啟動；在下一次執行開始之前，先檢查上一次是否已經完成。

Keep Every Result, and Bring the Keys Back

對每一次載入，保留輸入版本、轉換後的載入檔案、作業 ID、成功和失敗的結果及其錯誤、耗時、記錄數和對賬輸出。一次大型遷移之後，總會有人問「這條記錄怎麼了？」，我要的是證據，而不是「我覺得那是在週六上午的第三個 CSV 裡」。批次結果只保留七天，所以要邊做邊下載。以後再給這些工件定生命週期；在你仍然需要證明這次遷移的時候，不要把它們扔掉。

Then bring the keys back. The success results carry each record’s new Salesforce ID next to the key you sent. Write them back into the preparation data store, on the row that produced the record: that’s the last frame of module 1’s three sets of keys. On the Zendesk migration I described in module 1, that’s what made everything else possible. With the Salesforce IDs back on every row, I could load in phases (accounts and contacts first, then the cases, then the comments and files that hang off them), and each phase found its parents by the IDs the last one returned. I could do a partial load: just the rows that failed, or just one group. And I could reload: fix a transformation, regenerate the affected rows, and send only those, knowing exactly which Salesforce records they’d update. Without the keys back, every rerun is a guess. ⟲

失敗的多少也是一個「零的個數」的問題。100 條裡有 20 條失敗很糟糕；2,000 萬條裡有 20 條失敗依然需要調查，但它是一個不同的訊號。看比率，而不只是數量，並對失敗進行分類：驗證、缺少父記錄、重複的鍵、鎖定、資料類型、自動化、權限。分好類之後，它們向你展示的是架構問題，而不是一個巨大的錯誤檔案。

而且效能不只是每秒處理多少條記錄。「我們把吞吐量翻了一番。」什麼變了？鎖定、失敗或重試增加了嗎？下游的自動化是不是跟不上了，共用重新計算是不是積壓了，另一個整合是不是被淹沒了？整個切換真的變短了嗎？遷移不是在真空中執行的。

Test knowledge. The Bulk API’s success results return each record’s new Salesforce ID with the data you sent, and its failure results return each error; both are kept for seven days. Capture them, write the IDs back to the source or staging data, and you can load in phases, retry only the failures, and reconcile what landed. ⟲

測試整個流程

先載入客戶，再載入連絡人，再載入訂單，每一步都單獨測試過，這不叫測試過的遷移。我要的是一次完整的演練，從頭到尾，用同樣的順序、轉換、身分、自動化策略和對賬，在有代表性的資料量下進行。依賴關係會在步驟之間顯現：十分鐘的延遲讓下一次載入在錯誤的時間開始，一個意料之外的失敗率花掉幾個小時來修復，一次共用重新計算打亂了時間表。完整的執行告訴你遷移是否可行，而不只是每條命令是否可行。

If the rehearsal includes a performance or scale test, put Salesforce’s lead time in the plan. “All tests must be pre-approved. Unapproved testing is subject to throttling or blocking,” and for tests at 120 requests a second or below, the request goes in “at least one week before the testing date” (Salesforce Performance and Scale Test FAQs （在新標籤頁中開啟）, Salesforce Help, checked 2026-10-27). A rehearsal that gets throttled proves nothing about cutover weekend. ⟲

AI 可以寫指令碼。執行仍然由你負責。

AI is very good at generating transformation scripts, load orchestration, error parsers, reconciliation queries and retry logic. Use it. But generated code can be wrong at enormous speed: a wrong transformation on ten records is ten fixes; on ten million, the Number of Zeros applies to the mistake too. The way I use it is step by step: clean, transform and migrate steps, with every generated command and action tracked. That’s deterministic, auditable automation, not unattended judgment. Test it at realistic scale before you trust it with production data. ⟲

**考試之外：AI 改變了什麼。**AI 降低了構建遷移工具的工作量，也讓快速生成和執行批次操作變得更容易。操作規模越大，就越需要確定性的轉換、被追蹤的步驟、演練、控制和對賬。

架構師在這裡決定什麼

哪種載入機制和 API 版本適合工作的資料量和形狀，而不是把遷移當作事務性流量。
由依賴關係驅動的物件和關係載入順序，包括記錄和關係什麼時候需要分開處理。
Which operation each set of records gets (insert, update or upsert), based on what the preparation data store knows already exists. ⟲
Where parallel processing is safe, and where concentration, locking or automation justify sorting by parent, a model change, or serial processing. ⟲
哪些穩定的鍵、遷移身分、記錄所有權和可重複的工具，能讓演練和生產載入都是確定性的。
How results and new IDs come back, so loads can run in phases, retry only failures, and reload safely. ⟲
在生產規模下信任這個流程之前，需要什麼證據、失敗分類和完整的流程演練。

裝卸平台依然需要一個計劃

The loading dock moves enormous amounts of data. It doesn’t decide which truck unloads first, where the pallets go, which shipment belongs to which customer, what happens to a damaged pallet, whether two forklifts are using the same aisle, or whether everything that should have arrived did. That’s architecture. Know the Number of Zeros and the shape behind them. Load in dependency order, use stable keys, control concurrency, bring the keys back, make the process repeatable, keep the evidence, and rehearse the whole thing. Then use the loading dock as fast as the architecture safely allows. ⟲

延伸探索

Salesforce, Bulk API and Bulk API 2.0 Limits and Allocations （在新標籤頁中開啟）, Developer Limits and Allocations Quick Reference, checked 2026-11-01. ⟲
Salesforce, General Guidelines for Data Loads （在新標籤頁中開啟） and Limits （在新標籤頁中開啟）, Bulk API 2.0 and Bulk API Developer Guide, checked 2026-11-01. ⟲
Salesforce, Best Practices for Deployments with Large Data Volumes, Loading Data from the API （在新標籤頁中開啟）, Winter ‘27, checked 2026-11-01. ⟲
Salesforce, REST API Developer Guide, Create Records Using sObject Collections （在新標籤頁中開啟）, checked 2026-11-01. ⟲
Salesforce, SOAP API Developer Guide, create() （在新標籤頁中開啟）, and Bulk API 2.0, Create a Job （在新標籤頁中開啟）, checked 2026-11-01. ⟲
Salesforce, Bulk API 2.0 and Bulk API Developer Guide, JobInfo （在新標籤頁中開啟） (the Bulk API 1.0 job fields, concurrencyMode), API version 68.0 (Winter ‘27), checked 2026-11-01. ⟲
Salesforce Help, Salesforce Performance and Scale Test FAQs （在新標籤頁中開啟）, checked 2026-11-01. ⟲
Integration Architect 課程：Bulk (coming soon) 深入講解了 Bulk API。
本課程中：為規模而設計（熱點和所有權傾斜）。
Salesforce Developers, Designing Record Access for Enterprise Scale, Ownership Data Skew （在新標籤頁中開啟）, checked 2026-11-01. ⟲
Salesforce Developers, Salesforce CLI Command Reference, data upsert bulk （在新標籤頁中開啟）, checked 2026-11-01. ⟲