Trade-off Thinking

TL;DR: "It depends" is technically correct and completely useless unless you finish the sentence. Every design choice buys one property and pays with another. Senior engineers name both sides, commit with a reason tied to a specific constraint, and record what they gave up so their future selves can revisit. Jeff Bezos's 70% rule says most decisions should be made with about 70% of the information you wish you had^[1]. Waiting for 90% is almost always too slow.

Learning Objectives#

After this module, you will be able to:

• Finish every "it depends" with a named dependency, a threshold, and a recommendation
• Use the six-part articulation template (Context, Options, Chosen, Why, Given up, Reversible?) to structure any design decision
• Classify decisions as one-way or two-way doors and match process weight accordingly
• Identify cognitive biases (sunk cost, resume-driven, survivorship, NIH) that corrupt architecture decisions
• Apply Kent Beck's 3X model to recognize when a correct past decision becomes wrong today
• Write a lightweight Architecture Decision Record that survives team turnover

Intuition#

You are buying a car. You want safety, fuel economy, cargo space, acceleration, and a low monthly payment. You cannot have all five at their maximum. A minivan maximizes cargo and safety but sacrifices acceleration. A sports car maximizes acceleration but sacrifices cargo and cost. A compact sedan optimizes fuel economy and cost but gives up cargo and acceleration.

Nobody calls this "hard." You name what matters most for your life right now, you pick the car that fits, and you accept what you gave up. You do not pretend the minivan is also fast.

System design works the same way. You have consistency, availability, latency, throughput, cost, simplicity, and developer velocity. You cannot max all of them. The engineer's job is to name which ones matter most for this system at this scale, pick the architecture that fits, and say out loud what was sacrificed.

The rest of this chapter teaches you how to do that naming, picking, and saying out loud in a way that builds trust, survives team turnover, and sounds senior in an interview.

Theory#

What a trade-off actually is#

A trade-off is a design decision where improving one measurable property of the system necessarily degrades another. The word "necessarily" matters. If a choice improves property X at no cost to anything else, it is not a trade-off but a free lunch, and in a mature design review the first response to a free-lunch claim is to look for the hidden cost.

The mechanism is physical, economic, or algorithmic:

• Physical: The speed of light caps intercontinental round-trip time to roughly 80 to 150 ms (e.g., US-to-Europe ~80 ms, US-to-Asia ~150 ms), so strong consistency across regions buys latency.
• Algorithmic: Every index that speeds reads slows writes, because each write updates both the row and the index.
• Economic: Hiring PostgreSQL operators is cheaper than hiring Spanner-equivalent operators, so self-hosted strong consistency buys operational cost.

Fred Brooks's "No Silver Bullet" (1986) gives the meta-category: essential complexity is what the problem demands; accidental complexity is what the tools force on you^[2]. You can trade accidental complexity for accidental complexity all day (pick Kafka or RabbitMQ; pick Go or Rust) and the system gets no better. You only move forward by trading accidental cost for essential progress.

The primary axes#

Most system design decisions move along a small set of recurring dimensions:

The right way to say "it depends"#

"It depends" is correct. It is also the verbal equivalent of a blank stare. The fix is a three-part sentence:

1. Name the dependency. "It depends on whether we need cross-region consistency."
2. State the branches. "If yes, use synchronous replication and accept 150 ms write latency. If no, use async replication and accept stale reads up to 2 seconds."
3. Recommend for this context. "Given our SLO of 200 ms writes and single-region deployment, I would start with synchronous replication."

This is the six-part articulation template compressed into speech. The full written version, distilled from Michael Nygard's ADR format^[3] and Bezos's reversibility doctrine^[1:1]:

The six-part articulation template: every architecture decision fits this shape. If you cannot fill "Given up," you have not thought hard enough.

Use this template in ADRs, design docs, interview answers, and Slack threads. It is the single most senior-sounding verbal shape in engineering.

Cognitive biases that ruin decisions#

Humans are bad at trade-off analysis under pressure. Kahneman's "Thinking, Fast and Slow" catalogs the System 1 heuristics that hijack design reviews^[4]:

• Sunk cost. "We have already spent 6 months on this migration, we cannot stop now." Yes you can. The 6 months are gone regardless.
• Availability bias. The most recent incident dominates risk estimates. A team that just had a Kafka outage overweights "Kafka is unreliable" even if the root cause was misconfiguration.
• Confirmation bias. You seek evidence for the option you already like. The engineer who wants to use Rust finds benchmarks; the one who wants Go finds hiring data.
• Resume-driven development. Engineers pick technology that advances their careers, not their systems. Kelsey Hightower argued publicly that Kubernetes is "a platform for building platforms," not an endgame, a point often cited in discussions of teams adopting it for resume value on workloads that do not need it^[5].

The antidote is institutional: Architecture Decision Records force written, delayed commitment^[3:1]. Pre-mortems force you to imagine failure before you ship. Weighted decision matrices force you to make the argument about weights (a discussable topic) rather than about options (an opinion).

One-way vs two-way doors#

Bezos introduced the one-way/two-way door taxonomy in his 2015 letter to shareholders, where he called them "Type 1" (consequential, irreversible) and "Type 2" (changeable, reversible) decisions^[6]. In his 2016 letter he revisits the concept more memorably: "Many decisions are reversible, two-way doors. Those decisions can use a light-weight process. For those, so what if you're wrong?"^[1:2] He pairs this with the 70% rule and "disagree and commit" as a tool for unblocking two-way-door decisions under disagreement.

The failure mode he describes: large organizations defaulting to the heavyweight Type 1 process for decisions that are actually reversible Type 2 doors, resulting in "slowness, unthoughtful risk aversion, failure to experiment sufficiently, and consequently diminished invention."^[6:1]

Bezos's reversibility test: match process weight to decision reversibility. Two-way doors get a Slack vote; one-way doors get an ADR or RFD.

For engineers: choosing a programming language for a new service is usually a one-way door (rewrite cost is huge). Choosing a cache eviction policy is usually a two-way door (change the config, restart). Choosing a database engine is somewhere in between, and the skill is recognizing which side it falls on for your specific system.

The honesty test#

Dan McKinley's "Choose Boring Technology" (2015) gives every team a budget of roughly three "innovation tokens"^[7]. Each new piece of technology spends one. The benefit of a new tool is usually advertised loudly; the operational cost (maintenance, 3 AM on-call, unknown-unknowns) is invisible at adoption time and crushing in year two.

McKinley's line: "The grim paradox of this law of software is that you should probably be using the tool that you hate the most. You hate it because you know the most about it."^[7:1]

The honesty test: for every pro you list, you must also state its corresponding con. If your design doc has 7 pros and 0 cons for your preferred option, you are writing a pitch deck, not an architecture decision.

The dimension-counting anti-pattern is what happens when teams decide by counting bullets rather than weighing them. Seven pros vs five cons does not mean the seven-pro option wins. One con ("we lose all data during a region failure") can outweigh seven pros. Use a weighted decision matrix to expose the real disagreement: about weights, not about options^[8].

When trade-offs change over time#

Kent Beck's 3X model names three stages of product evolution, each with different optimal trade-offs^[9]:

Explore, Expand, Extract: the same technical trade-off can be correct in one phase and wrong in the next. Discord's Mongo-to-Cassandra-to-ScyllaDB path is a canonical instance.

• Explore: Low scale, high uncertainty. Optimize for learning speed. Accept messy code, ship experiments, use the tool you know.
• Expand: The product works and growth exposes bottlenecks. Engineering effort shifts to removing rate-limiting resources.
• Extract: Mature phase. Optimize for efficiency, profitability, and operational stability.

A trade-off that maximized learning speed in Explore (hand-rolled Rails app, one database) becomes a liability in Expand when the growth curve is non-linear. The 3X model gives vocabulary for "the right call last year is the wrong call now" without anyone being wrong^[9:1].

Real-World Example#

Discord: Cassandra to ScyllaDB (2017 to 2023)

Discord's message storage evolved through all three 3X phases, and each transition was a trade-off revisited^[10].

Explore (2015 to 2017): MongoDB replica set. Fast to build, familiar to the team. Gave up: horizontal write scaling.

Expand (2017 to 2022): Migrated to Cassandra with replication factor 3. Grew from 12 nodes to 177 nodes storing trillions of messages. The trade-off bought: write throughput and horizontal scaling. The trade-off paid: JVM garbage collection pauses caused latency spikes (p99 of 40 to 125 ms for historical reads), required manual node reboots, and created "gossip dance" operational toil during compaction^[10:1].

Extract (2022 to 2023): Migrated to 72 ScyllaDB nodes, each with 9 TB disk. Results: p99 historical reads dropped to 15 ms; p99 inserts held steady at 5 ms; cluster size dropped from 177 to 72 nodes^[10:2].

The key engineering decisions illustrate trade-off thinking:

1. Kept the data model, changed the engine. Cassandra Query Language compatibility made this a two-way door at the schema layer (application code barely changed) even though it was a one-way door at the ops layer (new C++ database, new failure modes).
2. Rewrote the Spark migrator in Rust. The initial Spark-based migration estimated 3 months. The team spent engineering time building a Rust migrator that finished in 9 days at 3.2 million messages per second^[10:3]. Classic trade-off: dev time vs wall-clock time-to-value.
3. Accepted the one-way door of abandoning a JVM-GC-prone database for a shard-per-core C++ one, motivated by years of on-call toil.

What Discord gave up: JVM ecosystem compatibility, Cassandra's larger community, and the team's existing operational muscle memory. What they gained: 3x latency improvement, 60% fewer nodes, and elimination of GC-induced incidents.

Discord's migration decision mapped onto the "When NOT to change" framework. Every gate passed with a specific, measurable answer.

Trade-offs#

These rows describe decision-making approaches, not substitutable alternatives. Readers pick by door type and context, the same axis the one-way/two-way flowchart earlier in this chapter encodes. Drop the "Our Pick" framing; each row is already a when-to-use recommendation.

The approach is a function of reversibility and stakes, not preference. Match process weight to the door type and to how much blast-radius a wrong answer carries.

Common Pitfalls#

Exercise#

Design Challenge: Your team operates a Python monolith on EC2. It serves 10k RPS at p99 of 600 ms (SLO: 300 ms). A junior engineer proposes rewriting it in Go microservices on Kubernetes. Senior leadership asks for your recommendation. Present the trade-offs using the six-part articulation template and give a defensible recommendation.

Key Takeaways#

• Every "it depends" needs a named dependency, a threshold, and a recommendation. Finish the sentence.
• Use the six-part template (Context, Options, Chosen, Why, Given up, Reversible?) for every significant decision. It fits in an ADR, a Slack message, or a 30-second verbal answer.
• Classify decisions as one-way or two-way doors. Two-way doors get fast, lightweight process. One-way doors get deliberate, written process^[1:5].
• Name what you gave up. If your design doc has only pros, you are writing marketing, not engineering.
• Cognitive biases (sunk cost, resume-driven, survivorship, confirmation) are organizational facts, not moral failings. Counter them with written artifacts and explicit weights.
• Trade-offs change over time. Beck's 3X model (Explore, Expand, Extract) gives vocabulary for "the right call last year is the wrong call now" without blame^[9:2].
• Sometimes the best decision is the null decision: keep what you have. "Technology we operate at 3 AM" is usually worth more than "technology that is new to us."

Further Reading#

• Choose Boring Technology (Dan McKinley, 2015) - The essay that named innovation tokens; read it once a year as a calibration exercise against novelty bias.
• Documenting Architecture Decisions (Michael Nygard, 2011) - The original ADR post; 1,200 words that changed how teams record decisions.
• 2016 Letter to Shareholders (Jeff Bezos) - Source of the one-way/two-way door taxonomy and the 70% rule.
• How Discord Stores Trillions of Messages (Bo Ingram, 2023) - The clearest single post on a multi-year infrastructure trade-off revisited with specific numbers.
• RFD 1: Requests for Discussion (Oxide Computer, 2020) - A working RFD process embedded in a public company's engineering culture; extends ADRs with a lifecycle.
• An Elegant Puzzle (Will Larson, Stripe Press, 2019) - Engineering-manager framing of trade-offs, weighted matrices, and when to formalize decisions.
• Under Deconstruction: Shopify's Monolith (Philip Muller, 2020) - The modular monolith as a third option between "microservices" and "ball of mud"; 2.8M lines of Ruby, still one deploy^[11].
• Spanner, TrueTime and the CAP Theorem (Eric Brewer, 2017) - The author of CAP revisiting his own theorem in light of a system that trades dollars (atomic clocks) for stronger consistency.
• Fast/Slow in 3X: Explore/Expand/Extract (Kent Beck, 2018) - The three-phase model for recognizing when a correct past decision becomes wrong today.

Flashcards#

References#