What Is System Design, Really?

Let me start with a story.

In 2012, Instagram was acquired by Facebook for $1 billion. At the time, Instagram had 13 employees. They were serving 30 million users with a tiny team, on infrastructure they’d built themselves.

Around the same time, a startup with 50 engineers and a similar product launched. Within months of any real traffic, their system was on fire. Outages every day. Performance disasters. Pages that took 20 seconds to load.

What was different? Instagram wasn’t using magic technology. They weren’t using a different programming language. They weren’t smarter people.

The difference was that Instagram’s engineers had thought carefully about constraints — the real, physical limits of what computers and networks can do — and designed their system to live comfortably inside those limits. The other startup treated their software like it existed in an infinitely fast, infinitely reliable world. It didn’t.

That’s what system design is. It’s the practice of building software that respects the laws of physics.

The Physical Reality Your Code Lives Inside

Here’s something computer science courses often skim over: your software runs on physical hardware in a physical world. That sounds obvious, but its implications are profound.

Consider these numbers. Take a moment to really sit with them:

That’s not a difference in kind — it’s a difference of 100,000x between your CPU cache and your disk. A billion times between your CPU and an overseas network request.

Now think about what a “simple” database query actually does:

1. Your application code runs (nanoseconds)
2. It opens a network connection to the database (milliseconds)
3. The database reads some rows from disk (milliseconds per read)
4. The database sends results back over the network (milliseconds)

You can write the most beautiful, algorithmically perfect code in the world. But if it’s making five database round-trips where one would do, or reading a full million-row table when an index would narrow it to ten rows — the physics wins. Your code is slow.

Key insight: System design is, at its core, the art of arranging software and data so that most operations stay in the “fast” part of this table, and the expensive operations happen as rarely as possible.

What “Designing a System” Actually Means

When people say “system design,” they might mean anything from “draw the boxes and arrows for this feature” to “figure out how this needs to work for the next five years.”

But fundamentally, designing a system means making a set of explicit choices about:

Where does state live? State is data that persists over time — user accounts, messages, transactions. Every piece of state has to live somewhere: in memory (fast, temporary), on disk (slow, durable), in a database (structured, queryable), in a cache (fast, potentially stale). Choosing where state lives has enormous consequences.

How do components communicate? Do services talk to each other synchronously (one calls another and waits for a response) or asynchronously (one sends a message and moves on)? Synchronous is simple but brittle. Asynchronous is flexible but complex. There’s no free lunch.

What happens when things fail? Not if — when. Your database will have a bad day. A network switch will hiccup. A bug will appear in production. The question is whether your system fails catastrophically or degrades gracefully.

What are you optimizing for? Every system has to choose. Low latency or high throughput? Strong consistency or high availability? Low cost or high performance? You genuinely cannot have all of them at once, and the right choice depends entirely on what the system actually needs to do.

These aren’t abstract concerns. They’re the specific decisions that separate a system that hums along reliably from one that pages engineers at 3am.

The Trade-Off Trap

Here’s the uncomfortable truth that takes engineers years to really internalize:

There are no solutions. There are only trade-offs.

Want low latency? Add a cache. But now you might serve stale data, and you have to manage cache invalidation (notoriously one of the hardest problems in computer science).

Want high availability? Replicate your data across multiple machines. But now you have to deal with replication lag — two replicas might have slightly different data for a moment.

Want to handle 1 million requests per second? Distribute your load across 100 servers. But now coordination between those servers becomes its own problem.

Every architectural decision buys you something and costs you something else. The engineer who says “why don’t we just…” is usually ignoring what that change will cost.

This isn’t a reason to be paralyzed. It’s a reason to be deliberate. To write down what you’re optimizing for, explicitly, before you start designing. To make trade-offs consciously instead of accidentally.

A Mental Model That Actually Works

Here’s a way of thinking about any system that I’ve found genuinely useful:

Think of a system as a set of resources with bottlenecks between them.

Resources are the things a system uses: CPU, memory, disk I/O, network bandwidth, database connections. Bottlenecks are the places where demand exceeds capacity — the place where requests pile up waiting.

Every performance problem is a bottleneck. Your job as a system designer is to:

1. Identify where the bottleneck actually is (not where you think it is — measurement is required)
2. Decide whether to eliminate it (make that resource faster/bigger) or move it (shift load somewhere else)
3. Understand what new bottleneck your change will create

When Instagram was scaling, their main bottleneck was the database. A single database server could only handle so many queries. Their solution wasn’t “buy a bigger database” — it was to put a caching layer in front of it (Redis), so most read requests never touched the database at all. They moved the work to a faster resource.

But that created a new question: what happens when the cache is empty? When data is missing from cache and everyone goes to the database at once? (This is called a “thundering herd” — you’ll learn about it in the Caching track.) Moving bottlenecks is the central game of scaling.

Before You Design Anything: The Questions You Must Answer

Here’s a practical framework not for interviews, but for actually designing systems. Before you draw a single box:

What is this actually for? Who uses it? What’s the single most important thing it must do? What’s the consequence if that thing fails?

What does the workload look like? Is it read-heavy or write-heavy? Does it get steady traffic or massive spikes? How much data does it store, and how fast does that grow? These numbers determine almost everything.

What are the non-negotiables? Can you afford 500ms latency, or does it need to be under 50ms? Can you tolerate data being slightly stale, or must every read reflect the latest write? Can you lose any data in a crash, or must everything be durable?

What can you accept failing? This is the most important and most neglected question. Every system has components. If component X fails, what happens to the user experience? If component Y is slow, what degrades? Answering this gives you your fault tolerance requirements.

You might notice that none of these questions mention specific technologies. Databases, message queues, caches, microservices — those are all answers to questions. You need the questions first.

The URL Shortener: A Concrete Starting Point

Let’s make this concrete with an example you’ll see throughout this platform: a URL shortening service (like bit.ly).

What it does: Takes a long URL, returns a short 7-character code. When someone visits the short URL, it redirects them to the original.

The workload profile:

• 1000 new short URLs created per second
• 100,000 redirects per second (reads heavily outnumber writes)
• URLs accessed immediately after creation need to work instantly

The non-negotiables:

• Redirects must be fast (users will notice anything over 100ms)
• A URL once created must always work (data durability)
• The service needs to stay up even if individual machines fail

What can fail:

• If the “create new URL” service is slow, that’s annoying but not catastrophic
• If the “redirect” service fails, every existing short link on the internet breaks — that’s catastrophic

Now look what this analysis tells you before you’ve designed anything:

• The redirect path is far more critical than the create path
• 100x more reads than writes means a cache is almost certainly necessary
• Durability is required (can’t lose URLs)
• High availability on the redirect path specifically

These requirements almost dictate the architecture. You’d put a read cache in front of the database for redirects. You’d store URLs in a durable database. You’d probably separate the write path (URL creation) from the read path (redirects) so they can scale independently.

You’d discover problems as you went: how do you generate unique 7-character IDs across multiple servers without collisions? (You’ll learn about this in the distributed systems track.) How do you keep the cache fresh? (Caching track.) How do you make redirects globally fast? (Networking track.)

This is system design as it actually works — not a pattern you apply, but a series of problems you discover by asking the right questions.

What This Platform Is For

Every track in this platform is about building genuine understanding, not memorizing patterns.

Understanding why indexes exist (not just what they are). Understanding why distributed consensus is hard (not just what Raft is). Understanding why caching invalidation is famously difficult (not just the LRU algorithm).

The goal is that after going through these lessons, you should be able to look at an architecture you’ve never seen and make reasonable guesses about why each piece is there, what problems it’s solving, and what would happen if you removed it.

That’s the skill. Not “name five database sharding strategies.” But understanding, deeply enough that you could derive those strategies yourself from first principles.

Use the simulator above to try building the URL shortener — make choices and watch what constraints you run into. Then continue to How Computers Work, where we’ll build the foundation that makes everything else click.