Pull or Push
Ce contenu n’est pas encore disponible dans votre langue.
Module 1 in the chapter The Doors In, chapter 4 of the Data 360 Architect course. Each module stands on its own. Next: Connectors Hold the Credentials
The question this module answers: How do you choose the door for a source: does Data 360 pull it, or do you push it?
There are a lot of ways to get data into Data 360. Before you look at any of them, ask one question about the source: where does the data already live? The answer usually picks the door for you.
When Data 360 pulls
Section titled “When Data 360 pulls”Data 360 can set up connectors and reach out on a schedule to get your data. You don’t send anything; it comes and collects. It can pull from:
- Your Salesforce orgs, one or many, through the Salesforce CRM connector. That’s chapter 5.
- B2C Commerce and Marketing Cloud, through their own connectors.
- Cloud storage: Amazon S3, Microsoft Azure Blob Storage and Google Cloud Storage. Put a file there, and Data 360 picks it up on the stream’s schedule.
With pull, Data 360 owns the schedule. The connection, with its credentials, is set up once (module 2). And there’s no code of yours in the middle that someone has to keep running. The pilots file in chapter 1 came in this way: a file in S3, pulled by Data 360.
When you push
Section titled “When you push”If the data isn’t in one of those places, you have two choices. Put it somewhere Data 360 can pull from, usually cloud storage. Or switch to a push, where your systems send the data in:
- SFTP: push CSV files to a secure file transfer server, and a stream picks them up.
- The Ingestion API, bulk: large files in a job, for big batches (module 4).
- The Ingestion API, streaming: small JSON payloads, sent as things happen (module 4).
- The web and mobile SDKs: your websites and apps send behavior as people use them (chapter 13).
Push gives you control of the timing. Flights are pushed: in my own project, a flight departs and the event goes to the Ingestion API’s streaming endpoint. In this course you’ll push a month of real flights from SkyData the bulk way (module 4), and replay them as a stream in chapter 13. In my own tests, streamed data arrived about every three minutes, and Salesforce documents the same: streamed data “is processed asynchronously approximately every 3 minutes.” That’s a different world from a file pulled once an hour.
The cost of push is that it’s yours. Your code builds the payload, your code handles the errors, your code retries. Somebody has to own it after the project ends.
Or don’t move it at all
Section titled “Or don’t move it at all”When I first recorded my walk through these doors, I hadn’t found a way to bring Snowflake data in except by exporting it through one of the other doors. That has changed. Salesforce now lists inbound zero copy: Data 360 queries data where it lives, in platforms such as Snowflake and Google BigQuery, without copying it in. That’s a third answer to “pull or push”: sometimes, neither. Chapter 15 is all about it.
Salesforce’s patterns, as a decision table
Section titled “Salesforce’s patterns, as a decision table”Salesforce’s architects describe the inbound doors as patterns, each with the problem it solves. Here they are, with where this course teaches each one:
| Salesforce’s inbound pattern | Pull or push | Where it’s taught |
|---|---|---|
| Bulk data ingestion from cloud storage | Pull | Chapter 4, module 3 |
| Bulk data ingestion from Salesforce clouds | Pull | Chapter 5 |
| Near real-time CRM synchronization via streaming | Pull | Chapters 5 and 13 |
| Event-driven ingestion via the Ingestion API, streaming | Push | Chapter 4, module 4 |
| Real-time web and mobile behavior ingestion | Push | Chapter 13 |
| Event stream ingestion from Kinesis and MSK | Pull, from your event platform | Chapter 13 |
| Inbound zero copy | Neither: queried in place | Chapter 15 |
From Salesforce Architects, Data 360 Integration Patterns.
The order of questions
Section titled “The order of questions”Put together, it’s a short list of questions, asked in order:
- Where does the data live today? If it’s somewhere Data 360 can pull from, pull.
- If not, can it land in cloud storage? A file dropped in S3 on a schedule is often the simplest door of all.
- Does it need to arrive as it happens? Then push it, with streaming, and plan who owns that code.
- Does it need to come in at all? If it lives in a lakehouse and you only need to query it, look at zero copy first.
Watch it on video
Section titled “Watch it on video”In 2024 I filmed this same walk through the doors, source by source, with a help link for each. The note under the video says what’s still right and what’s changed since, including the door I hadn’t found then: zero copy.
What an architect decides here
Section titled “What an architect decides here”- The door for each source, and the reason, written down beside it in the dictionary from chapter 3.
- Who owns the code for every push, after the project is over.
- Which sources need to arrive as they happen, and which can wait for the next pull. Streaming is billed by the row in both credit models, at many times the cost of batch (module 4 has the numbers), so “as they happen” needs a business reason.
Explore
Section titled “Explore”All checked 2026-11-14.
- Get Started with Ingestion API (opens in a new tab), Salesforce Developers: streaming latency.
- Data 360 Integration Patterns (opens in a new tab), Salesforce Architects: the inbound, outbound and zero copy patterns.