콘텐츠로 이동

The Doors In

이 콘텐츠는 아직 번역되지 않았습니다.

Chapter 3 of Data 360 Architect, Part 1: Bringing Data In. Previous: Understanding Your Data Structures.

Chapter 2 was about knowing your data before it comes in. Now it comes in. There are a lot of ways to get data into Data 360, and each one decides how fresh the data is, who holds the credentials, and who owns the code when something fails. This chapter starts with the one question that picks the door, then walks the two doors you’ll use most: Amazon S3, where Data 360 pulls, and the Ingestion API, where you push.

  • Start with where the data lives. If Data 360 can pull from it, pull. If not, put it in cloud storage or push it. And sometimes, with zero copy, don’t move it at all.
  • Credentials live in a connector, set up once. The people who hold the secrets set up the connection; the people who map the data pick it. Every environment needs its own.
  • Refresh mode is a business question. If a record is missing from the file, should it disappear? Yes: full refresh. No: upsert.
  • Pick the slowest schedule the use case allows, and none at all in practice orgs. A sandbox spends production’s credits.
  • Streaming is near real time, not real time. Salesforce processes streamed data about every 3 minutes. Big loads go to bulk, the loading dock.
  • Validate before you load, and own every line, even the lines an AI assistant drafted for you.

Five modules, each standing on its own.

  1. Pull or Push (coming soon). The question that picks the door, and Salesforce’s inbound patterns as a decision table.
  2. Connectors Hold the Credentials (coming soon). Like a named credential: set up once, picked by many, and set up again in every environment.
  3. The Pull Door: Amazon S3 (coming soon). Which files, upsert or full refresh, and how often.
  4. The Push Door: the Ingestion API (coming soon). Define, stream, push; streaming or bulk; validate first.
  5. The Other Doors, Named (coming soon). Every other way in, and the chapter that teaches it.
  • The door for each source, and the reason, written in the dictionary from chapter 2.
  • Who holds the credentials and sets up each connector, in every environment.
  • The refresh mode and schedule for each stream.
  • Who owns the code for every push, after the project ends.

All checked 2026-11-01.

Next: Bringing In CRM Data