Keeping Another System in Step

🌐 Cũng có bằng 19 ngôn ngữ khác

English · Español · 日本語 · Português (Brasil) · Français · Deutsch · Italiano · 한국어 · 简体中文 · Português (Portugal) · 繁體中文 · Bahasa Indonesia · Türkçe · Polski · ไทย · Українська · Español (México) · Français (Canada) · Deutsch (Schweiz)

Hình ảnh được tạo bằng AI

Được dịch bằng AI. Cộng đồng chưa xem xét. Đọc bằng tiếng Anh Phần này đã thay đổi trong bản tiếng Anh và chưa được dịch lại. (57)

Mô-đun 5 trong 5 của chương Chuyển đổi và đồng bộ dữ liệu, chương 10 của khóa học Data Architect. Mỗi mô-đun có thể đọc độc lập. Trước: Cuối tuần chuyển giao · Tiếp theo: chương sau, Giao dịch, xử lý đồng thời và độ tin cậy

The question this module answers: How do you get a copy of Salesforce data into another system, at volume, and keep it current? ⟲

So far we’ve talked about getting data into Salesforce. Data architects also have to get data out. Not to leave Salesforce: because the data lives in Salesforce, and another system needs a copy of it. A warehouse such as Snowflake for reporting and analytics, an archive or a backup, another Salesforce org, or a sandbox with realistic data for a rehearsal. The Number of Zeros works in reverse too: 500 records out is easy; 50 million out is architecture. And the copy is rarely a one-time thing. Usually the other system wants it today, and again tomorrow, and every day after that. ⟲

Start With the Picture

Here’s Steve Trucking’s landscape after cutover. RouteDesk is retired. Salesforce is the system of record for every delivery. And one arrow carries the status history out of Salesforce, through the integration layer, into Snowflake, where leadership’s five-year reports and the analysts live. ⟲

Nhấp để phóng to

Salesforce keeps the data. Snowflake keeps a mirror of it, every day. ⟲

Salesforce isn’t going anywhere. Snowflake doesn’t replace it; Snowflake needs a copy of part of it, kept in step. That’s what getting data out is for, and it’s two problems in a row: the first copy, which can be big, and every copy after it, which has to be current. ⟲

Why You’d Pull a Lot Out

The first copy is the big one. When Snowflake starts mirroring status history, it needs everything so far, not just today’s changes. At Steve Trucking that starts right after cutover: module 3 deferred the status history of RouteDesk’s 196,400 migrated Work Orders to “one backfill after the load.” That backfill is the first big extract out of Salesforce. And as the business grows, every new mirror, every new report store, every rebuild after a problem starts with a copy of everything. ⟲

Other reasons bring you here too: an archive for records that leave the core (chapter 7), a backup you can restore from, another Salesforce org that needs a slice of the data, a sandbox with realistic data for a rehearsal. ⟲

Và đừng xuất mọi thứ chỉ vì bạn có thể. Tám năm lịch sử, tệp lớn, bản ghi lỗi thời, dữ liệu tạm thời, giá trị dẫn xuất, giao dịch đã lưu trữ: hãy hiểu vì sao mỗi loại được trích xuất. Mỗi bản ghi đều tốn chi phí về thời gian, lưu trữ, chuyển đổi, di chuyển, xác thực, đối chiếu, bảo mật và nghĩa vụ lưu giữ ở phía sau. “Xuất toàn bộ org” không tự động là một kiến trúc.

Những lối ra

Salesforce cho bạn vài lối ra, và mỗi lối hợp với một việc khác nhau.

• Data Export Service. Setup’s Data Export produces a zip of CSV files for the whole org, weekly (Enterprise, Performance and Unlimited Editions) or monthly (all editions except Database.com), on demand or on a schedule. You can run it manually once every 7 days (weekly) or 29 days (monthly). Professional and Developer Editions get the monthly export only. The files are deleted 48 hours after the email is sent, not counting weekends; formula and roll-up summary fields are always excluded; and whoever has the Weekly Data Export permission can see all exported data (Export Backup Data from Salesforce (mở trong thẻ mới), Salesforce Help, checked 2026-10-27). It’s a backup-shaped snapshot, not a migration pipeline. ⟲
• Data Loader và báo cáo. Data Loader xuất từng đối tượng một ra CSV, với Export All để bao gồm cả bản ghi đã xóa và đã lưu trữ; xuất báo cáo hợp với một bản trích xuất nhỏ, đã được định hình. Cả hai đều ổn cho hàng nghìn dòng, không phải hàng chục triệu.
• Bulk query. For volume, the loading dock works in both directions: sometimes the truck is unloading, sometimes it’s being loaded. A bulk query runs asynchronously and returns its results in files. Bulk API 2.0 manages the batching for you, and on large tables it chunks the query itself: its query jobs “automatically determine the best way to divide your query job into smaller chunks, helping to avoid failures or timeouts” (Understanding Bulk API 2.0 Query (mở trong thẻ mới), checked 2026-10-27). On Bulk API (1.0) you turn that on yourself, with PK chunking. Salesforce’s Large Data Volumes guide draws the line: “When using a query that can return more than one million results, consider using the query capability of the Bulk API 2.0” (Extracting Data from the API (mở trong thẻ mới), Winter ‘27, checked 2026-10-27). ⟲
• Một sản phẩm sao lưu để khôi phục, như Backup & Recover của chính Salesforce, làm một việc khác với chuyển đổi: khôi phục về một thời điểm, không phải chuyển dữ liệu đến ngôi nhà mới.

For a mirror, I want the extract repeatable, logged and reconcilable, so that usually means a bulk query driven by a script, not clicks. Steve Trucking’s backfill is exactly that: a bulk query of the migrated Work Orders and their status history, written to Snowflake once, and reconciled. ⟲

Test knowledge. Know the ways data comes out of Salesforce and what each is for. The Data Export Service is a scheduled, whole-org CSV backup (weekly or monthly by edition; files kept 48 hours; no formula or roll-up fields). Data Loader and reports suit small, object-by-object or shaped extracts. Bulk API query handles volume asynchronously. A backup product restores a point in time; it isn’t a migration tool. ⟲

Very Large Copies: PK Chunking

Fifty million records as one giant query may never finish. PK chunking divides the table by its primary key, the record ID: Salesforce adds ID boundaries to your query and runs it as a series of smaller queries, each a separate batch with its own results. The default chunk is 100,000 records and the maximum 250,000. Salesforce recommends it for tables of more than about 10 million records, or when a bulk query keeps timing out. That’s where I use it too: it’s specialized, for very large exports. It’s a Bulk API 1.0 header, it works on a defined list of objects, and each chunk counts toward the daily batch allowance (PK Chunking (mở trong thẻ mới), checked 2026-10-27). Bulk API 2.0 does the same chunking for you, but an ORDER BY or a LIMIT switches it off on either API, and the query may then time out. ⟲

Steve Trucking will get there. About a million Work Orders a year, growing 30 percent a year, adds up to about 12.8 million by the end of year 6, unless chapter 7’s archiving keeps the table smaller. Work Order is on Salesforce’s list of supported objects. ⟲

Nhấp để phóng to

Twelve million rows become about 52 queries of 250,000. ⟲

Cú pháp thuộc về tài liệu. Điều kiến trúc sư nên nhận ra là khuôn mẫu: khi Số chữ số 0 thay đổi, chiến lược trích xuất có thể cũng phải thay đổi.

Test knowledge. For very large extracts (Salesforce’s line is more than 10 million records), PK chunking splits a bulk query into ranges of record IDs, each run and downloaded separately. It’s a header you set on Bulk API 1.0; Bulk API 2.0 query jobs chunk automatically. ORDER BY and LIMIT disable it. ⟲

Then Keep It Current

The first copy is a snapshot. By the time it finishes, the source has changed: crews closed deliveries, coordinators changed windows, partners revised manifests. That’s normal. The architecture has to answer: what changed after our baseline? That’s the job of a watermark: a timestamp, a sequence, a change-tracking mechanism, a source identifier. Conceptually it says: this is what we copied, this is the point through which it was current, these are the changes since. Without that boundary, delta processing is guesswork. On the Salesforce side, the watermark is usually SystemModstamp, which is indexed (chapter 8): each run asks for what changed since the last run’s watermark, and moves the watermark forward only when the run succeeds. ⟲

Don’t forget the deletes. A record deleted in Salesforce must disappear from the mirror too, or the reports keep counting it. A normal query doesn’t return deleted records; a bulk query with the queryAll operation “returns records that have been deleted because of a merge or delete” (Create a Query Job (mở trong thẻ mới), Bulk API 2.0, checked 2026-10-27). But deleted records stay in the Recycle Bin for 15 days at most, less if someone empties it (Deleting Data (mở trong thẻ mới), Best Practices for Deployments with Large Data Volumes, checked 2026-10-27), so a mirror that looks for deletes less often than that will miss some. And a hard delete skips the Recycle Bin altogether, so plan how the mirror learns about those too. ⟲

How often? That’s chapter 9’s question, and Salesforce’s own guidance gives the line: the SOAP API’s getUpdated() and getDeleted() calls for intervals over 5 minutes, and for anything sooner, let Salesforce send the change (Asking Every Minute). Steve Trucking does both: the backfill is a bulk query, and from the cutover watermark onward, Change Data Capture sends each status change through the integration layer as it happens. ⟲

Nhấp để phóng to

Once: the first copy, everything so far. It sets the watermark. ⟲

Nhấp để phóng to

Then every change after the watermark, as it happens. ⟲

Nhấp để phóng to

And every few days, the deletes. One big copy, then a stream of small ones. ⟲

Test knowledge. To keep another system current, extract only what changed since a watermark (SystemModstamp is indexed), and use queryAll to catch records deleted or merged. Deleted records stay in the Recycle Bin for 15 days, so run the delete check more often than that. For changes needed within minutes, use events (Change Data Capture) instead of polling. ⟲

A Copy Once, or a Copy Forever

Đây là sự phân biệt tôi dùng: chuyển đổi dữ liệu có điểm bắt đầu và kết thúc xác định. Chúng ta đang đi từ một trạng thái sang một trạng thái khác. Nó có thể bắt đầu với lần trích xuất production đầu tiên, bao gồm vài lần tải trước, kéo dài một tuần, bao gồm vài lần tải phần thay đổi. Nó vẫn là chuyển đổi, vì có một đích đến: một thời điểm mà đợt chuyển đổi hoàn tất.

Đồng bộ là một quá trình liên tục để giữ hai hệ thống nhất quán. Không có chuyện “chuyển đổi xong, tắt đi”. Cả hai hệ thống vẫn còn ý nghĩa, thay đổi vẫn tiếp tục xảy ra, và chúng phải được di chuyển. Giờ chúng ta quay lại chương 9: hệ thống nào sở hữu giá trị, nó di chuyển theo hướng nào, khóa nào nhận diện nó, ai đẩy và ai bảo đảm tính đầy đủ, lỗi được thử lại và giám sát thế nào, chuyện gì xảy ra khi cả hai hệ thống cùng thay đổi một thứ? Khi yêu cầu là “giữ cho các hệ thống này nhất quán”, chúng ta không còn thiết kế một đợt chuyển đổi nữa. Chúng ta đang thiết kế tích hợp.

Steve Trucking had one of each, on the same weekend. RouteDesk into Salesforce was a migration: its delta loads closed the gap, then it ended. Salesforce to Snowflake is synchronization: it started that weekend, and it never ends. ⟲

Nhấp để phóng to

Same weekend, two arrows. One has an end, so everything about it is different. ⟲

Nhưng một lần tải phần thay đổi không tự động có nghĩa là đồng bộ. “Chúng tôi có các lần tải phần thay đổi, nên chúng tôi đang đồng bộ.” Không hẳn. Hãy hỏi: quy trình có điểm kết thúc xác định không? “Có, chúng tôi chạy các lần tải phần thay đổi này cho đến khi chuyển giao, rồi quy trình cũ dừng lại” vẫn là chuyển đổi; các lần tải phần thay đổi giúp chúng ta đạt đến trạng thái cuối của nó. “Không, cả hai hệ thống vẫn hoạt động và việc này giữ chúng nhất quán vô thời hạn” là đồng bộ. Thao tác có thể trông giống nhau. Nghĩa vụ kiến trúc thì khác.

Test knowledge. Migration has a defined end; synchronization is ongoing. Delta loads before a cutover belong to the migration. Keeping another system in step with Salesforce afterward is synchronization, and it needs an integration design with a clear owner for each value. ⟲

The Mirror Isn’t the Master

Salesforce owns the delivery status. Snowflake holds a copy. That has to stay true. Suppose an analyst notices a wrong status in Snowflake and corrects it there. Now the two systems disagree, and the next sync either overwrites the correction or, worse, somebody builds a feed to send it back. Then both systems think they own Delivery Status. Last write wins? Salesforce wins? Human review? If that’s unanswered, we haven’t built a mirror; we’ve created two writable copies. Chapter 9’s rule: technical permission isn’t architectural authority. Fix it at the source, and let the sync carry the fix. ⟲

So write the arrow’s contract down, the way chapter 9 does: who owns each value, which way it moves, what key identifies it, how fresh it must be, what counts as delivered, how failures are retried, how it’s monitored, what the watermark is. Those aren’t extract-script questions; they’re operating-model questions. The first copy builds the starting state. Synchronization keeps it coherent. ⟲

Prove the First Copy, Then Keep Proving It

Use module 4’s four proofs (counts, totals, spot checks, aggregate reports) on the first copy. Prove the baseline before synchronization becomes the normal operating model, or the sync will faithfully maintain a bad starting point. Integration doesn’t repair a bad copy just because it runs continuously; it can keep two systems perfectly in step around the wrong data. ⟲

Then keep proving it. A mirror that stops quietly is a data quality problem with a delay: the reports look fine and slowly go stale. Chapter 9’s proof of delivery applies to every run: the side that knows what should arrive checks that it did, with counts against the watermark window, and someone is told when it doesn’t. ⟲

AI Can Help Spot the Drift. Keep the Boundary Deterministic.

AI can help find changed records, unexpected differences between Salesforce and the mirror, anomalies, likely conflicts and records that seem to have been missed. Useful. But “is this change in the mirror yet?” shouldn’t depend on an AI’s guess from context. Watermarks, keys, ownership rules and state decide it. ⟲

Beyond the exam: what AI changes. AI can make comparison and anomaly detection between a system and its mirror much faster. Keep the sync deterministic: explicit keys, authority and watermarks decide what moves and when. ⟲

Kiến trúc sư quyết định gì ở đây

• Why each copy of Salesforce data exists, which classes of data it really needs, and which system owns each value. ⟲
• Cơ chế xuất nào phù hợp với công việc và Số chữ số 0: Data Export Service, Data Loader hoặc báo cáo, truy vấn hàng loạt, hay PK chunking cho các bảng rất lớn.
• What watermark establishes the baseline, how later changes and deletes are captured, and how often (polling or events). ⟲
• Whether a data movement is a migration with an end or a synchronization without one, and what integration contract keeps the mirror consistent. ⟲
• How the first copy is proven, and how every run after it is checked. ⟲

Synchronization Keeps You There

Getting data out isn’t leaving Salesforce. It’s giving another system what it needs: once at volume, then every day after, current, complete and never the master. The first copy is the big job; keeping it current is the long one. Remember the line from this chapter: migration has a defined start and end; synchronization is an ongoing process to keep two systems consistent. The first copy gets the mirror there. Synchronization keeps it there. ⟲

Tìm hiểu thêm

• Salesforce Help, Export Backup Data from Salesforce (mở trong thẻ mới), checked 2026-11-01. ⟲
• Salesforce, PK Chunking (mở trong thẻ mới), Understanding Bulk API 2.0 Query (mở trong thẻ mới), Create a Query Job (mở trong thẻ mới) and Bulk API and Bulk API 2.0 Limits and Allocations (mở trong thẻ mới), checked 2026-11-01. ⟲
• Salesforce, Best Practices for Deployments with Large Data Volumes, Deleting Data (mở trong thẻ mới), checked 2026-11-01. ⟲
• In this course: Integration and Data Movement (the contract, polling or events, proof of delivery) and, for off-core history, Designing for Scale. ⟲
• Salesforce Developers, Extracting Data from the API (mở trong thẻ mới), Best Practices for Deployments with Large Data Volumes, Winter ‘27, checked 2026-11-01. ⟲