Design a Ticketing System (BookMyShow / Ticketmaster)

TL;DR. Ticketing is the canonical "strong consistency for inventory, eventual consistency for everything else" problem. Live Nation processes 500 million tickets per year ^[1], but a single hot onsale can attract 14 million users against 70,000 seats ^[2]. The architecture linearizes one operation (seat reservation via Redis SET NX PX with Lua scripts), pushes everything else to cached reads and async fan-out, gates admission through a virtual waiting room, and coordinates payment through a saga with compensations. The pivotal trade-off: hold TTL duration versus payment-completion latency.

Learning Objectives#

After this module, you will be able to:

• Design a three-state seat-locking protocol (available, held, sold) that prevents double-booking under 100K TPS burst
• Choose between Redis single-node locks, Redlock, ZooKeeper, etcd, and DB row locks with explicit safety reasoning
• Apply the saga pattern to coordinate seat reservation with external payment providers
• Architect a virtual waiting room that converts a traffic spike into a bounded admission rate
• Defend anti-bot strategies (Verified Fan, device fingerprinting, CAPTCHA) with regulatory context
• Estimate capacity for a Ticketmaster-scale platform handling 3.5 billion daily requests at peak ^[1:1]

Intuition#

A ticketing system looks like a trivial CRUD app. Create an event, list seats, let users buy them. At 10 users it works fine. At 10 million users hitting "Buy" on the same 70,000 seats at 10:00:00 AM, it collapses, and the reason is contention.

Think of a bakery with 50 numbered tickets on the counter. One customer walks in, picks ticket #7, pays, leaves. No problem. Now open the doors to 5,000 people simultaneously. Everyone lunges for the counter. Two people grab ticket #7 at the same time. One tears it. The other claims they had it first. The baker has no record of who touched it when.

The engineering insight: you cannot let 5,000 people touch the counter at once. You need a bouncer at the door (the waiting room) who admits 10 people per minute. Inside, each ticket has a clip that only one hand can close (the atomic lock). And the baker does not hand over the pastry until the card clears (the saga), but holds the ticket for exactly 10 minutes so an abandoned attempt does not block the next buyer forever.

Three forces shape every decision: (1) strong consistency on the seat-to-buyer mapping is non-negotiable, (2) adversarial traffic from bots is not an edge case but the dominant load source, and (3) demand exceeding supply by 10x to 100x is the normal operating condition for hot events.

Requirements#

Clarifying Questions#

• Q: Are seats uniquely identified (assigned seating) or general admission? Assume: Both. Assigned seating is the hard case; general admission is a counter decrement.
• Q: What is the hold duration before payment must complete? Assume: 10 minutes. Must exceed p99 payment latency with margin.
• Q: Do we own the payment system or integrate externally? Assume: External (Stripe, Adyen). We cannot run 2PC across their systems.
• Q: What is the peak burst scenario? Assume: 500K concurrent users for a single event, 100K seat-selection TPS at peak ^[1:2].
• Q: Is bot traffic a design concern or an ops concern? Assume: Design concern. Bots generated 3x prior peak traffic during the 2022 Eras Tour onsale ^[3].
• Q: Multi-region? Assume: Yes for reads (event catalog, seat maps). Single-leader for inventory writes per event.

Functional Requirements#

• Browse events and view real-time seat availability maps
• Select and hold specific seats for a bounded duration (10 minutes)
• Complete purchase within the hold window; confirm seat as sold
• Release held seats automatically on TTL expiry or payment failure
• Queue users fairly when demand exceeds backend capacity
• Deliver e-tickets with rotating encrypted barcodes (SafeTix) ^[4]

Non-Functional Requirements#

• Load: 500M tickets/year steady state; 3.5B requests/day peak onsale ^[1:3]; 100K seat-lock TPS burst
• Latency: p99 < 500 ms for seat selection; payment confirmation < 30 seconds end-to-end
• Availability: 99.9% for the purchase path; 99.99% for catalog reads
• Consistency: Linearizable for seat state transitions; eventual for catalog and recommendations
• Durability: Zero tolerance for double-booking; zero ticket loss after payment confirmation

Capacity Estimation#

Key ratios: Read:write on the seat map is ~50:1 during browsing, collapsing to ~2:1 during active checkout. The waiting room absorbs 90%+ of inbound traffic before it reaches inventory. Redis single-node handles ~100K ops/sec ^[6], matching the burst target with one node per hot event.

API and Data Model#

API Design#

POST /v1/events/{event_id}/seats/hold
  Idempotency-Key: <uuid>
  Body: { "seat_ids": ["sec-A:row-12:seat-7"], "user_id": "u-abc" }
  Returns: 201 { "hold_id": "h-xyz", "held_until": "2026-05-04T10:10:00Z", "version": 1 }
  Errors: 409 seat already held, 429 rate limited, 403 not admitted

POST /v1/holds/{hold_id}/confirm
  Idempotency-Key: <uuid>
  Body: { "payment_intent_id": "pi_stripe_123" }
  Returns: 200 { "ticket_id": "t-final", "status": "sold" }
  Errors: 410 hold expired, 402 payment failed

DELETE /v1/holds/{hold_id}
  Returns: 204 (seat released)

GET /v1/events/{event_id}/seats?section=A
  Returns: 200 { "seats": [{"id": "...", "state": "available|held|sold"}], "updated_at": "..." }

Hold creation uses an idempotency key to prevent duplicate holds on retry. The version field is a fencing token for the confirm step. The seat-map GET is served from cache with 1-2 second staleness.

Data Model#

-- Source of truth (PostgreSQL)
CREATE TABLE seat (
  event_id   BIGINT  NOT NULL,
  seat_id    TEXT    NOT NULL,
  state      TEXT    NOT NULL DEFAULT 'available',
  holder_id  BIGINT,
  held_until TIMESTAMPTZ,
  version    BIGINT  NOT NULL DEFAULT 0,
  PRIMARY KEY (event_id, seat_id)
);

-- Atomic reserve (single statement, no SELECT FOR UPDATE needed)
UPDATE seat
SET state = 'held', holder_id = $1,
    held_until = now() + interval '10 minutes',
    version = version + 1
WHERE event_id = $2 AND seat_id = $3
  AND (state = 'available' OR (state = 'held' AND held_until < now()))
RETURNING version;

Redis serves as the hot-path lock layer for high-TPS events. PostgreSQL is the durable source of truth, reconciled asynchronously. Kafka streams hold/sale/release events to notifications, analytics, and the seat-map cache invalidator.

High-Level Architecture#

The waiting room gates admission at the edge; only admitted users reach the inventory service, which coordinates seat locks through Redis and persists to PostgreSQL via Kafka-driven reconciliation.

Write path: An admitted user selects seats. The inventory service executes SET seat:{event}:{id} {owner_token} NX PX 600000 on Redis ^[7]. On success, it publishes a seat.held event to Kafka and asynchronously writes to PostgreSQL. The saga orchestrator (Temporal) begins the reserve-charge-confirm workflow.

Read path: The seat-map service serves from a Redis-backed cache refreshed every 1-2 seconds via Kafka consumer. Browsers poll or receive SSE updates. The CDN caches the event catalog and venue metadata indefinitely until invalidated.

Async path: Kafka fans out hold/sale/expiry events to the notification service (email, SMS, push), the audit log (event-sourced for compliance), and analytics (real-time dashboards for ops).

Deep Dives#

Seat locking: the three-state machine#

A seat has exactly three states: available, held, and sold. A fourth state (refunded) transitions back to available after a relist decision.

A seat transitions atomically from available to held via Redis SETNX; TTL expiry or payment failure returns it to available without manual intervention.

The atomic transition uses Redis SET key val NX PX 600000 where the key encodes seat:{event_id}:{seat_id} and the value is a random owner token ^[7:1]. The NX flag ensures mutual exclusion. The PX 600000 sets a 10-minute TTL in milliseconds. Release uses a Lua script that checks ownership before deleting:

if redis.call('get', KEYS[1]) == ARGV[1] then
  return redis.call('del', KEYS[1])
else
  return 0
end

This prevents a delayed client from releasing a lock that expired and was re-acquired by another user ^[7:2]. The owner token is generated per-attempt; without it, client A could release client B's lock after a GC pause.

Why 10 minutes? The hold TTL must exceed p99 payment latency (typically 5-15 seconds for card auth) with generous margin for user hesitation, 3D Secure challenges, and network retries. Setting it to p50 payment latency guarantees race conditions at the boundary ^[8]. Setting it too long (30 minutes) blocks inventory from legitimate buyers.

The boundary race: If payment succeeds at T+9:58 but the hold expired at T+10:00, the confirm step must atomically verify the fencing token (version number) before transitioning to sold. If the seat was re-sold during the 2-second gap, the original buyer gets a refund ^[8:1].

Distributed lock alternatives and the Kleppmann debate#

The choice of lock mechanism sits on a cost/safety axis:

Martin Kleppmann's canonical critique of Redlock ^[9:1] argues it is "neither fish nor fowl": too expensive for efficiency locks, not safe enough for correctness locks. The core problem: Redlock lacks fencing tokens. A GC pause on client A causes its lease to expire; client B acquires the lock; client A resumes and writes without checking. The fix is to pair any lock with a monotonic fencing token that the storage layer validates on write ^[9:2].

For ticketing, the pragmatic choice is single-node Redis SET NX PX per hot event, paired with a PostgreSQL version column as the fencing token on the confirm path. Redis failover can cause a brief window of double-holds, but the confirm step's version check prevents double-sales. This is "efficiency lock + correctness fence" rather than relying on the lock alone for correctness.

Saga for reservation + payment#

Distributed Transactions explains why 2PC fails across independent services. Stripe cannot participate as a resource manager in a distributed transaction; you cannot PREPARE a card authorization ^[13]. The saga pattern is the industry-standard answer.

The saga reserves the seat, charges the card with a Stripe idempotency key, and either confirms or compensates; a crash at any step triggers the appropriate rollback via Temporal's durable workflow replay.

Idempotency keys are non-negotiable. Stripe holds the key-to-response mapping for 24 hours ^[14]. The key is generated server-side at reservation time as hash(user_id, seat_ids, attempt_nonce) and persisted with the saga state. Network retries reuse the same key, preventing duplicate charges.

Compensation is not rollback. A refund is not an "uncharge." It appears on the customer's statement, may take 5-10 business days, and can itself fail. The saga must handle compensation failures with escalation to a dead-letter queue and manual reconciliation ^[15].

Orchestrator choice: Temporal (the MIT-licensed successor to Uber's Cadence, built by the same creators) persists workflow state to Cassandra/PostgreSQL, survives process restarts, and provides built-in retry with exponential backoff ^[13:1]. AWS Step Functions is an alternative for teams already on AWS.

Virtual waiting room#

When 14 million users arrive at 10:00:00 for 70,000 seats, the backend cannot serve them all simultaneously. The waiting room converts a spike into a trickle.

The edge gateway checks a per-data-center admission budget; users over budget receive a queue cookie and poll until their bucket is admitted.

Cloudflare Waiting Room implements this with a two-tier Durable Object architecture ^[16]. Per-data-center Durable Objects aggregate local worker counts. A single Global Durable Object reconciles worldwide state. The admission budget is divided across data centers proportional to the previous minute's traffic ratio, so each data center knows its local limit without coordinating per-request ^[16:1].

Ticketmaster's Smart Queue works identically from the user perspective: sign in before onsale, receive a queue position when the sale starts, enter the purchase flow at a rate matched to backend capacity ^[17]. Queue-it handles tens of millions of requests per minute using DynamoDB at 100-200K TPS per table ^[5:1].

Estimated wait time = users_ahead / avg_admission_rate_per_minute ^[16:2]. Wildly wrong estimates destroy trust; Cloudflare uses the previous minute's actual admission rate, not a theoretical maximum.

Real-World Example#

Ticketmaster, 2022-11-15: The Taylor Swift Eras Tour Meltdown

The Eras Tour presale generated 3.5 billion system requests on a single day, 4x Ticketmaster's previous peak ^[1:6]. Pre-registration attracted 3.5 million Verified Fan signups (the largest in history), of which 1.5 million received presale codes ^[1:7]. Actual traffic during the onsale reached approximately 14 million users, including bots ^[2:2].

The failure was not in the seat-locking or inventory paths. It was in the Verified Fan code-validation service, which bots targeted "for the first time" ^[3:1]. Bot traffic was 3x Ticketmaster's prior peak ^[3:2]. The code-validation servers buckled, causing ~15% of interactions to error, including passcode failures that caused fans to lose carted tickets ^[1:8].

Ticketmaster responded by slowing the queue drain rate to stabilize the system. The trade-off: longer waits but fewer checkout errors ^[1:9]. General sale was canceled on 2022-11-17 due to insufficient remaining inventory ^[2:3]. Despite the chaos, over 2 million tickets sold on Ticketmaster on 2022-11-15, the most ever sold for a single artist in one day; total tour sales reached 2.4 million across Verified Fan and Capital One onsales on both Ticketmaster and SeatGeek ^[1:10].

Live Nation President and CFO Joe Berchtold testified before the US Senate Judiciary Committee on 2023-01-24, attributing the failure to "industrial scalpers" and unprecedented bot traffic ^{[3:3] [18]}. The hearing became a referendum on Live Nation/Ticketmaster's market dominance following the 2010 merger, not just the technical outage ^[18:1].

Antitrust aftermath: On 2024-05-23, the US Department of Justice and 30 state attorneys general filed United States v. Live Nation Entertainment in the Southern District of New York, alleging Live Nation illegally monopolized the live-events industry ^[19]. Trial began 2026-03-02. The DOJ settled with Live Nation on 2026-03-09 (no forced divestiture), but 33 states rejected the settlement and continued litigation ^[20]. On 2026-04-15, a federal jury found Live Nation and Ticketmaster liable for illegally maintaining monopoly power in ticketing and amphitheater markets ^[21]. Remedies (potentially including a forced Ticketmaster divestiture) remain pending before the court as of May 2026.

Key lesson: Anti-bot defenses must protect every endpoint on the critical path, not just the purchase endpoint. The attackers targeted the cheapest bottleneck (auth validation), not the hardest one (inventory).

Trade-offs#

The meta-decision: where to place the linearization boundary. Linearize only the seat state transition (Redis + fencing token). Push everything else (catalog, seat-map reads, notifications, analytics) to eventually-consistent cached paths. This minimizes the blast radius of the consistency requirement.

Scaling and Failure Modes#

At 10x load (5B tickets/year, 1M TPS burst): Single Redis node saturates. Mitigation: shard inventory by event_id; each hot event gets a dedicated Redis instance. The waiting room absorbs the multiplied edge traffic without architectural change since it runs on edge compute ^[16:4].

At 100x load (50B tickets/year, 10M TPS burst): The saga orchestrator becomes the bottleneck (workflow state persistence). Mitigation: partition Temporal namespaces by event; use event-local orchestrators that share nothing. Payment provider rate limits become binding; negotiate dedicated capacity or fan out across multiple providers.

At 1000x load: The architecture shifts to a CDN-first model where the seat map is a static asset updated via invalidation, and the only origin-bound request is the atomic lock acquisition. The waiting room becomes the primary user interface, not a gate.

Failure modes:

• Redis node crash during onsale: Held seats in Redis are lost. PostgreSQL version column prevents double-sales on confirm. Affected users see "hold expired" and must re-select. Blast radius: one event's in-flight holds (seconds of data).
• Payment provider timeout: Saga retries with the same idempotency key ^[14:1]. After 3 retries, the saga releases the hold and notifies the user. The seat returns to available within seconds, not 10 minutes.
• Waiting room cookie replay attack: Attacker harvests admitted cookies and replays them from multiple clients. Mitigation: bind the cookie to a device fingerprint and IP range; invalidate on mismatch. Rate-limit per-cookie request volume.

Common Pitfalls#

Follow-up Questions#

Exercise#

Exercise 1: Hold expiration edge cases#

Design the behavior when a user's card declines at T+9:55 inside a 10-minute hold. Should you extend the hold? What if the same user retries with a different card at T+10:02? What if clock drift between the Redis TTL and the saga orchestrator causes disagreement about whether the hold is still valid?

Key Takeaways#

• Double-booking is a credibility-ending failure. Optimize for zero double-sales before optimizing latency, throughput, or UX. The fencing token on confirm is the last line of defense.
• The saga pattern is always the right answer for reservation + payment. 2PC looks correct but fails because external payment providers are not resource managers ^[13:3].
• A waiting room is not user-hostile; it is the only path to fair access. Without it, the fastest bots win and the backend collapses under thundering-herd load ^[16:5].
• Idempotency keys on payment are non-negotiable. Every retry without the same key is a potential duplicate charge ^[14:3].
• Anti-bot is an architecture concern, not an ops afterthought. The 2022 Eras Tour failed because bots targeted the auth path, not the inventory path ^[3:5].
• Hold TTL must exceed p99 payment latency, not p50. The boundary race between hold expiry and payment confirmation is the most common source of oversell bugs ^[8:3].

Further Reading#

• How to do distributed locking. Kleppmann's canonical critique of Redlock with the fencing-token argument; required reading before choosing any distributed lock.
• Distributed Locks with Redis. Salvatore Sanfilippo's Redlock specification; read alongside Kleppmann for the full debate.
• Building Waiting Room on Workers and Durable Objects. Cloudflare's two-tier Durable Object design showing how to divide a global admission budget without per-request coordination.
• Designing robust and predictable APIs with idempotency. Stripe's engineering blog on idempotency keys, exponential backoff, and safe retries for payment APIs.
• Ticketmaster: Taylor Swift Eras Tour Onsale Explained. Ticketmaster's own post-mortem of the 2022 failure with specific numbers on bot traffic and error rates.
• Saga pattern. Chris Richardson's reference definition of the saga pattern with orchestration vs choreography trade-offs.
• Temporal trip-booking tutorial. A worked example of saga with compensation in a durable workflow engine.
• BOTS Act of 2016. The federal statute making ticket-bot circumvention illegal under FTC enforcement.

Flashcards#

References#