Steve's Number of Zeros

Module 3 of 6 in the chapter Understanding Integration Requirements, chapter 6 of the Integration Architect course. Each module stands on its own. Previous: Listening for the Shape of the Integration · Next: Classifying the Data

The question this module answers: How do you get the volumes out of the business when nobody wants to commit to a number, and what do you turn them into?

Volume planning is crucial. Are we going to move ten transactions, or two? Are we going to move ten transactions of five gigabytes each, or not? The answer changes the design, the tools, the licenses and the cost. And it’s the one question the business never wants to answer.

When Nobody Will Give You a Number

Ask a client how many accounts they have, and they won’t want to admit to a number. “Well, it’s going to grow.” So I start throwing numbers at them. Ten? A hundred? A thousand? Ten billion? I get a little ridiculous, and then they finally say, “Okay, a hundred thousand.”

That’s Steve’s Number of Zeros. You don’t need the exact number. You need the number of zeros: ten, a hundred, a thousand, a million. People who can’t commit to 87,412 will happily tell you it’s closer to a hundred thousand than a million.

The method, step by step:

1. Get the business to agree to baseline numbers they clearly understand and will commit to.
2. If they can’t agree on an accurate number, pick one with just the number of zeros, and aim slightly higher if they can’t commit.
3. Turn them into the units an integration is designed for.

Here’s the classic. A hundred thousand accounts. How many opportunities do they close a month, per account? Two. How many line items per opportunity? One? A thousand? They zero in on ten.

They don’t sound like big numbers. But a hundred thousand accounts with two opportunities a month is 2.4 million opportunities a year, and with ten line items each, 24 million line items. That’s year one. If they keep five years of history, it’s 120 million.

Four Units

The integration team, and whoever’s selling you the integration layer, will always want to get you to one number: transactions per second. That’s one of four I want for every flow:

• Transactions per second (TPS): how many calls or messages.
• Records per second: a transaction can carry more than one record.
• Bytes per second: how much data actually moves.
• The activity window: how many hours a day this really happens in.

The way I get there: calculate the amount in a month, get it down to a day, and then narrow it to the window of activity, because we don’t assume everything’s running at two in the morning.

Back to the opportunities. Two hundred thousand a month, over 22 working days, is about 9,100 a day. If they only close them during an eight-hour business day in one time zone, that’s 28,800 seconds, so about 0.32 TPS: a third of a transaction a second. With ten line items each, that’s eleven records per transaction, about 3.5 records a second. Gentle. Now change one number, five hundred thousand accounts instead of a hundred thousand, and every number goes up five times. That’s why you get the zeros right.

Count Transactions and Bytes Separately

Now Steve Trucking. About 3,300 deliveries a day, each with about eight status updates: 26,400 updates a day. The crews work about twelve hours, so that’s about 0.6 a second. An easy integration, by transactions.

But each delivery also has about four photos, about 13,200 a day, at about 4 MB each. That’s about 53 GB a day, about 1.2 MB every second of the working day. So this solution has to handle about half a transaction a second, but it has to do some serious data moving for those photos.

Click to enlarge

Same method, different profiles.

Now turn it around. What if a chain of 1,000 stores sends three messages for every appointment, the day before, the day of and on arrival, with 100 appointments per store per day? That’s 300,000 messages a day. Over an eleven-hour day, about 7.6 a second: more than ten times Steve Trucking’s status updates, but each message is small.

One more thing the photos teach. Three photos at 4 MB each don’t arrive spread across a delivery’s messages; they ride on one of them, the “job completed” message, and that one is 12 MB. The average sizes the day. The largest message sizes the mechanism, because some mechanisms have a limit on how big one message can be. Chapter 11 checks those limits.

A lot of data with low TPS. High TPS with little data. They can have completely different behavioral profiles, and they lead to different designs. Size every flow by transactions and by bytes, separately.

People Are Traffic Too

The hardest numbers to get are the ones people make. “Our agents work cases, and every change goes to the other system.” How often? They can never tell you what a human does.

So I sit there and say: I’m a human in front of the computer. How long does it take me to open a case, type, type, type, and hit save? Thirty seconds? Longer. Let’s say five minutes. That’s twelve saves an hour. Five hundred agents, working eight hours a day in one time zone: 500 × 12 ÷ 3,600 seconds is about 1.7 TPS.

Even five hundred people typing away can generate real traffic, when the integration sends every change near real time.

The Activity Window

All of those per-second numbers are smoothed: the day’s volume spread evenly over the activity window. But the data doesn’t arrive evenly. It comes in bursts. You may have this much in a 24-hour day, but you need to move it in two hours.

So the window is the lever. Shrink it, and the same day gets denser.

Choose the window the way the business really works. Case agents in one time zone: their eight-hour working day is the window. Appointment messages for stores on both coasts: an eight-hour day in the first time zone, plus three more hours for the last one to close, is an eleven-hour window.

Click to enlarge

The window is the peak.

Then ask when the real bursts happen. If half of Steve Trucking’s photos arrive in a three-hour afternoon wave, that’s about 2.4 MB a second, double the smoothed rate. Crews coming back into signal after a dead zone send everything at once. Each peak needs its own window.

And plan for your highest season, not your average one. Let the other seasons dip down. In retail, the weekend everyone plans for is Black Friday: that’s the ultimate seasonality. Whatever the business’s Black Friday is, find it, and size for it.

When the peak is higher than the other system can take, that’s where queues and workers come in: something in the middle that takes the burst and feeds it on at a rate the other side can handle. Chapters 11 and 13 build them.

Then Three Years Out

Every one of these numbers is for today, and nobody wants an integration that only works for today. In chapter 1, we grew Steve Trucking at my usual baseline of 30% a year: about 2.2 times today’s volume by year three (The Building Nobody Described). Run the same method on year three: about 58,000 status updates a day, about 1.3 a second over twelve hours, and about 116 GB of photos a day.

That’s the number you design for.

Do Your Math Early

The key thing: calculate the integration traffic early. It decides how big the integration layer has to be, which mechanisms will fit, and which limits you’ll meet. Do your math early, and work it out, with the business in the room, one number at a time. Change one assumption, and watch which result moves. The Steve’s Number of Zeros for Integrations reference page has the method, four worked examples and a calculator you can use in that meeting.

What an Architect Decides Here

• For every flow: the business’s numbers, or at least their number of zeros, agreed and written down.
• Transactions, records and bytes per second, and the activity window each one is spread over.
• Which flows are heavy by transactions and which by bytes, and that each is designed for its own profile.
• The peaks: the busiest window of the day, the busiest season of the year, and the bursts.
• Year three’s numbers, at an agreed growth rate.

Beyond the exam: what AI changes. AI can do this arithmetic instantly, and it will happily guess the zeros for you if you ask. Don’t let it. A guessed number nobody in the business has agreed to is worse than no number, because it looks like a decision. Use AI to run the what-ifs once the business has committed. And add a line for AI’s own traffic: an agent that looks something up every time someone asks it a question is traffic too, and it’s the hardest kind to guess, because nobody knows yet how often people will ask.

Explore

• SteveTechArc, Steve’s Number of Zeros for Integrations: the method, four worked examples, and a calculator.
• The Building Nobody Described: the three-year horizon, and 30% a year.
• In the Data Architect course, the same zeros sized for storage and queries: Designing for Scale.