Networking Fundamentals for System Design

TL;DR: Every distributed system is a network problem in disguise. A first connection to a server across the Atlantic costs 3 round trips (DNS + TCP + TLS) before a single byte of application data flows, roughly 300 ms of dead air ^[1]. Master TCP vs UDP, the three HTTP generations, DNS, TLS 1.3, and the three real-time transports, and 80% of "design X" questions become tractable.

Learning Objectives#

After this module, you will be able to:

• Explain what each OSI layer does and where it shows up in real infrastructure
• Choose between TCP and UDP for a given workload
• Describe head-of-line blocking and how HTTP/2 and HTTP/3 address it
• Walk through a DNS lookup from your laptop to the authoritative server
• Sketch a TLS 1.3 handshake and explain when 0-RTT is safe
• Pick the right real-time transport: WebSocket, SSE, or long polling

Intuition#

Imagine you are mailing a package overseas. You do not think about the cargo ship, the customs office, or the sorting facility. You hand it to the post office, write an address, and it arrives. The internet works the same way: layers of software and hardware collaborate invisibly so your browser can say "page loaded."

But here is the catch. Every layer adds time. The post office stamps your package (DNS lookup). Customs verifies your identity (TLS handshake). The ship waits for a full container before sailing (TCP congestion window). Stack these delays on a first connection, New York to London, and you get roughly 300 ms of silence before a single useful byte moves. This module teaches you where that time goes, why some protocols feel instant while others lag, and how to pick the right transport for each job.

Theory#

The OSI model (practical view)#

The OSI model has 7 layers. In practice, system designers touch four:

The TCP/IP model collapses these into 4 layers (Application, Transport, Internet, Link). Most engineers use the two interchangeably. When someone says "layer 7 load balancer," they mean a balancer that parses HTTP and can route on path, headers, or cookies. AWS ALB is layer 7 with a default idle timeout of 60 seconds ^[2]. A "layer 4 load balancer" only sees the TCP 5-tuple (source IP/port, destination IP/port, protocol) and forwards raw bytes. AWS NLB is layer 4 with a default TCP idle timeout of 350 seconds ^[3].

TCP vs UDP#

TCP gives you a reliable, ordered byte stream. It handshakes, retransmits lost packets, and paces itself to avoid congesting the network. UDP gives you a datagram: fire it and forget.

TCP costs 1 round trip of dead air before any application byte flows; UDP skips this entirely.

On a 100 ms transcontinental link, that handshake is 100 ms of silence. The theoretical minimum round trip between New York and London through subsea fiber is about 56 to 62 ms (light travels at roughly 204,000 km/s in glass, two-thirds the speed of light in vacuum) ^[4]. Real-world transatlantic round trips sit around 70 to 90 ms ^[5].

Use TCP when correctness matters more than speed: HTTP, database connections, file transfer, SSH.

Use UDP when loss is tolerable or you want to build your own reliability: DNS queries (one packet, retry if it drops), video calls (a dropped frame beats a stalled stream), game networking, and QUIC (which reinvents reliable transport on top of UDP).

HTTP/1.1, HTTP/2, HTTP/3#

Three generations of the web's core protocol, each fixing the previous generation's bottleneck.

HTTP/1.1 (RFC 9112) is text-based and serial: one request at a time per TCP connection. Browsers work around this by opening up to 6 parallel connections per origin ^[6]. You still see HTTP/1.1 everywhere: internal services, curl scripts, health checks.

HTTP/2 (RFC 9113) multiplexes many requests over a single TCP connection using binary framing and HPACK header compression ^[7]. One connection, many parallel streams. The catch: TCP head-of-line (HOL) blocking. If one TCP segment drops, every HTTP/2 stream stalls until the retransmit arrives, even streams that had nothing to do with the lost packet ^[1:1].

HTTP/3 (RFC 9114) runs on QUIC (RFC 9000), which runs on UDP. QUIC builds its own streams with independent loss recovery, so a lost packet only stalls the stream it belonged to ^[8]. QUIC also combines the transport and TLS handshakes into a single 1-RTT operation and supports 0-RTT resumption where a returning client sends data in the very first packet.

HTTP/2 multiplexes over a single TCP stream (susceptible to TCP head-of-line blocking); HTTP/3 gives each stream its own loss recovery via QUIC over UDP.

As of May 2023, Cloudflare reported HTTP/3 carried about 28% of their global traffic, HTTP/2 about 64%, and HTTP/1.1 about 9% ^[9]. Chrome Mobile already sent roughly 40% of requests over HTTP/3 ^[9:1]. By 2025, Cloudflare Radar showed HTTP/3 at 21% of requests to their network, HTTP/2 at 50%, and HTTP/1.x at 29% ^[10]; the methodology differs from the 2023 blog (requests to Cloudflare vs traffic served by Cloudflare), so direct comparison requires caution.

DNS resolution#

DNS translates names like api.stripe.com into IP addresses. The lookup walks a hierarchy of caches.

Every DNS lookup is a hierarchical cache walk; caches at every layer mean popular names resolve in under a millisecond from warm cache.

Your OS ships a tiny stub resolver (the thing gethostbyname calls). It forwards to a recursive resolver like Cloudflare 1.1.1.1 or your ISP's server. The recursive resolver does an iterative walk: root, then TLD, then authoritative nameserver. Caches live at every layer, keyed by TTL.

Cloudflare's 1.1.1.1 resolver handles approximately 67 million DNS queries per second (authoritative plus resolver combined) across its Anycast network ^[10:1]. DNS runs over UDP on port 53 for queries under 512 bytes and falls back to TCP for larger responses. DNS-over-HTTPS (DoH) and DNS-over-TLS (DoT) add privacy by encrypting queries.

TLS 1.3 handshake#

TLS 1.3 (RFC 8446) cut the handshake from 2 round trips (TLS 1.2) to 1 round trip ^[11]. On a mobile network where each RTT costs 50 to 200 ms, that savings is significant.

TLS 1.3 cut the handshake to 1 RTT by letting the client ship its key_share in the first message; 0-RTT resumption adds early data to that flight but is replayable.

The client guesses the server's preferred key exchange group and sends its public key in the very first message. The server responds with its own public key, certificate, and Finished, all encrypted after the ServerHello. One round trip, done.

For repeat visits, TLS 1.3 supports 0-RTT resumption: the client sends application data in the very first packet using a pre-shared key from the previous session. This is fast but dangerous. 0-RTT data is not forward secret, and an attacker who captures the early data can replay it to multiple server instances ^[11:1]. Cloudflare does not enable 0-RTT by default and automatically rejects non-idempotent methods (POST, PUT) over 0-RTT ^[12].

Real-time transports: WebSockets, SSE, long polling#

HTTP is request-response. For server-pushed data (chat messages, stock ticks, notifications) you need a long-lived channel. Three options exist:

Long polling opens an HTTP request and the server holds it until data is ready or a timeout fires. Simple, works through any proxy. High overhead: every message is a fresh HTTP request with full headers.

Server-Sent Events (SSE) uses a long-lived HTTP response with Content-Type: text/event-stream. The server streams text events. One-way (server to client), built-in reconnection via Last-Event-ID, works over HTTP/1.1 and HTTP/2. Under HTTP/1.1, browsers cap at 6 concurrent connections per domain, so SSE consumes one of those slots ^[6:1]. Over HTTP/2 the limit becomes the negotiated SETTINGS_MAX_CONCURRENT_STREAMS; RFC 9113 sets no initial limit but recommends a value no smaller than 100; most servers (Kestrel, nghttp2, curl) advertise exactly 100.

WebSockets (RFC 6455) upgrade an HTTP connection into a full-duplex binary channel. Low per-message overhead: 2 to 14 bytes of framing ^[13]. Use for chat, multiplayer games, and collaborative editors where both sides push data.

Default to SSE for server-to-client streams; WebSockets only when the client also pushes; long polling only as universal fallback.

The opinionated recommendation: Default to SSE. It handles 90% of real-time use cases (dashboards, notifications, LLM token streams) with zero sticky-session complexity. Reach for WebSockets only when the client sends frequent application data (chat input, game state, cursor positions). Use long polling only when you must support environments that block everything else.

Real-World Example#

Cloudflare rolled out HTTP/3 across its global network starting September 2019, shipping support while the QUIC spec was still in draft status ^[1:2]. Their edge stack uses their own open-source QUIC library (quiche), sits behind Anycast BGP, and migrated congestion control from New Reno to CUBIC for large transfers.

In their April 2020 production benchmark, HTTP/3 delivered a time-to-first-byte (TTFB) of 176 ms on average versus 201 ms for HTTP/2, a 12.4% improvement ^[1:3]. For a 15 KB page load, HTTP/3 completed in 443 ms versus 458 ms for HTTP/2. The gains come primarily from eliminating the extra TLS round trip (QUIC combines transport and crypto handshakes) and from per-stream loss recovery on lossy mobile links.

By May 2023, HTTP/3 carried about 28% of Cloudflare's global traffic (up from near zero in 2019), with Chrome Mobile at roughly 40% ^[9:2]. As of late 2025, Cloudflare handles approximately 81 million HTTP requests per second on average, peaking at over 129 million ^[10:2].

The key lesson: HTTP/3's biggest wins appear on lossy, high-latency mobile networks where TCP head-of-line blocking hurts most. On fast wired links, the difference is marginal. Always advertise HTTP/2 as a fallback via Alt-Svc headers, because some enterprise firewalls still block UDP outbound on port 443.

Trade-offs#

Common Pitfalls#

Exercise#

Design Challenge: You are building a live sports score app for 10 million concurrent users. Scores update every few seconds during a match. Latency should be under 1 second. Design the client-to-server transport. Which protocol do you pick, and why? How do you handle reconnects, scaling, and cold starts?

Key Takeaways#

• TCP trades latency for reliability; UDP trades reliability for latency. QUIC gives you both by building reliable streams on top of UDP.
• A first HTTPS connection costs 3 RTTs (DNS + TCP + TLS 1.2). TLS 1.3 saves 1 RTT; HTTP/3 combines transport and crypto into 1 RTT total.
• HTTP/2 fixed HTTP/1.1's one-request-per-connection problem but inherited TCP's head-of-line blocking. HTTP/3 eliminates it with per-stream loss recovery.
• DNS is a hierarchical cache-first system. Short TTLs enable fast failover; long TTLs reduce resolver load.
• 0-RTT resumption sends data in the first packet but is replay-vulnerable. Never use it for non-idempotent requests.
• For real-time server push, default to SSE. Use WebSockets only when the client also pushes. Long polling is a last resort.
• Cloudflare's HTTP/3 rollout showed 12.4% TTFB improvement, with the biggest gains on lossy mobile networks.

Further Reading#

• RFC 9114: HTTP/3 - The definitive HTTP-over-QUIC spec; read sections 1 to 3 for the motivation and stream mapping.
• RFC 8446: TLS 1.3 - Sections 2 and 4 for the handshake; Appendix E for the security analysis of 0-RTT.
• Comparing HTTP/3 vs HTTP/2 Performance - Cloudflare's 2020 production benchmark with TTFB numbers, page load sweeps, and CUBIC vs BBR analysis.
• Even faster connection establishment with QUIC 0-RTT - Why Cloudflare disables 0-RTT by default and how they handle replay protection.
• How Discord Scaled Elixir to 5,000,000 Concurrent Users - Real WebSocket fan-out at scale: 500K sessions per Elixir VM, custom batching, and cascading failure patterns.
• MDN: EventSource - Authoritative reference on SSE browser behavior, the 6-connection limit, and Last-Event-ID semantics.
• The QUIC Transport Protocol: Design and Internet-Scale Deployment - Google's SIGCOMM 2017 paper on deploying QUIC at 7% of internet traffic.
• DDIA Chapter 8 (Designing Data-Intensive Applications, Martin Kleppmann), pp. 274 to 287 - "The Trouble with Distributed Systems" on unreliable networks and timing assumptions.

Flashcards#

References#