Design a Multi-Tenant SaaS Platform

TL;DR. A multi-tenant SaaS platform hosts 50,000 customer organizations on shared infrastructure while enforcing per-tenant isolation, tiered SLAs (99.5% free to 99.99% enterprise), and exactly-once metered billing. The pivotal decision is where each tenant sits on the pool-to-silo spectrum: pool (shared DB with Postgres RLS) for cost efficiency, bridge (schema-per-tenant) for moderate isolation, and silo (dedicated cluster) for enterprise compliance^[1]. Salesforce runs hundreds of thousands of orgs on a shared multitenant database with governor limits^[2]; Shopify isolates millions of stores in fully independent pods^[3]. The cardinal sin is cross-tenant data leakage, a missing WHERE tenant_id = ? that returns another customer's data.

Learning Objectives#

• Design a tenant-isolation architecture that maps silo, bridge, and pool models to pricing tiers and unit economics
• Implement tenant routing with tenant_id propagation that survives async hops, cache layers, and background jobs
• Build a metering pipeline achieving >99.99% billing accuracy using idempotent, exactly-once event semantics
• Contain noisy neighbors with per-tenant quotas across CPU, connections, and API rate limits
• Justify when to promote a "whale" tenant from pool to silo with zero-downtime migration
• Prove GDPR-compliant deletion via crypto-shredding for append-only stores

Intuition#

A multi-tenant SaaS looks like a trivial CRUD app with a tenant_id column. It handles 10 tenants fine. At 50,000 tenants spanning a 200,000x range in data volume (5 MB free trial to 1 TB enterprise), the architecture collapses in three ways.

First, isolation: a single missing tenant filter in a query, cache key, or background job leaks one customer's data to another. Unlike a single-tenant bug that shows too much of your own data, a multi-tenant leak shows someone else's data. That is an immediate SOC 2 finding, contract termination, and potential regulatory action^[4].

Second, economics: the top 10% of tenants drive 80% of load. If you silo every tenant, you pay for 50,000 idle database clusters. If you pool everything, one enterprise's batch import saturates the shared connection pool and degrades service for 49,999 others.

Third, billing: every API call, storage byte, and compute second becomes a line item on an invoice. A 0.1% metering drift on $100M ARR is $100K/year of invisible revenue leak or customer overcharge^[5].

The insight that unlocks the design: tenancy is a spectrum, not a switch. Run all three isolation models simultaneously. Free tenants land in the pool (cheap, shared, RLS-enforced). Pro tenants get bridge isolation (schema-per-tenant). Enterprise whales get silo (dedicated infrastructure). Price the spectrum. Let tenants promote themselves by paying more, or auto-promote them when their load threatens the pool.

Requirements#

Clarifying Questions#

• Q: Greenfield or retrofit? Assume: Greenfield. Retrofitting multi-tenancy onto a single-tenant system is harder but uses the same primitives.

• Q: Pricing model? Assume: Hybrid. Seat-based base fee plus usage-based overage (API calls, storage). Stripe Billing integration.

• Q: Compliance targets? Assume: SOC 2 baseline for all tiers. HIPAA and data residency available on enterprise tier only.

• Q: Data residency? Assume: EU tenants pinned to EU region. US default. Enterprise can choose region at signup.

• Q: Noisy-neighbor guarantee? Assume: Enterprise tier gets guaranteed isolation (silo). Pool tiers get best-effort with hard rate limits.

• Q: Tenant customization? Assume: Custom fields via metadata (no runtime DDL). White-label branding on enterprise tier.

Functional Requirements#

• Tenant signup with self-service provisioning (free/pro) and sales-assisted onboarding (enterprise)
• Tenant-scoped resource management: users, data, API keys, audit logs
• Per-tenant usage metering feeding automated invoicing
• Tenant admin controls: invite users, assign roles, view usage dashboard
• Tenant deletion with cascading cleanup and GDPR-grade erasure within 30 days^[6]

Non-Functional Requirements#

• Tenants: 50,000 active, ranging from 1-user free trials to 100K-user enterprises
• Load: 50K req/sec peak (business hours); 14K sustained
• Latency: p50 < 50 ms, p99 < 200 ms for tenant-scoped reads
• Availability: 99.5% (free), 99.9% (pro), 99.99% (enterprise)
• Billing accuracy: >99.99% (drift < 0.01% per billing period)
• Isolation: zero cross-tenant data leakage; enforced at database layer, not application trust

Capacity Estimation#

Key ratios:

• Pareto distribution: 10% of tenants generate 80% of load. Design quotas around the 90th percentile, not the mean.
• Read:write ratio: ~10:1 for typical SaaS workloads; metering is append-only at 7K events/sec.
• Tenant router cache hit rate: must exceed 99.5% to avoid becoming a bottleneck^[7].

API and Data Model#

API Design#

POST /v1/tenants
  Body: { "name": "Acme Corp", "tier": "pro", "region": "eu-west-1" }
  Returns: 201 { "tenant_id": "t_abc123", "status": "provisioning" }

GET /v1/tenants/{tenant_id}
  Returns: 200 { "tenant_id": "...", "tier": "pro", "usage": {...}, "quota": {...} }

POST /v1/tenants/{tenant_id}/users
  Body: { "email": "alice@acme.com", "role": "admin" }
  Returns: 201 { "user_id": "u_xyz", "invite_sent": true }

GET /v1/tenants/{tenant_id}/usage?month=2026-05
  Returns: 200 { "api_calls": 1420000, "storage_gb": 42.3, "billable_total": "$847.20" }

DELETE /v1/tenants/{tenant_id}
  Returns: 202 { "deletion_job_id": "del_789", "estimated_completion": "2026-06-03" }
  Side-effect: async erasure across all stores within 30 days

Context propagation: every authenticated request carries tenant_id in JWT claims. The API gateway extracts it, validates against the route, and sets X-Tenant-Id header for downstream services. Middleware rejects any request reaching the data layer without a tenant context.

Data Model#

-- Tenant metadata (PostgreSQL, control plane)
CREATE TABLE tenants (
  tenant_id   UUID PRIMARY KEY,
  name        TEXT NOT NULL,
  tier        TEXT NOT NULL CHECK (tier IN ('free','pro','enterprise')),
  region      TEXT NOT NULL,
  status      TEXT NOT NULL DEFAULT 'provisioning',
  shard_id    INT,
  created_at  TIMESTAMPTZ DEFAULT NOW()
);

-- Pool model: shared tables with RLS
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;
ALTER TABLE orders FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON orders
  USING (tenant_id = current_setting('app.tenant_id')::uuid)
  WITH CHECK (tenant_id = current_setting('app.tenant_id')::uuid);

Core entities: a tenant owns users, API keys, and generates usage events that feed the billing pipeline.

High-Level Architecture#

Every request flows through the tenant router, which resolves the tier and shard, enforces quotas, and emits usage events into the metering pipeline.

Write path: A tenant's API request hits the edge load balancer, which terminates TLS and forwards to the tenant router. The router extracts tenant_id from the JWT, looks up the tenant's tier and shard assignment in a hot cache (Redis, >99.5% hit rate), enforces the per-tenant rate limit, then routes to the appropriate data plane (pool, bridge, or silo). The data plane sets SET LOCAL app.tenant_id = ? before executing any query, activating RLS policies^[8][9].

Read path: Identical routing. For pool tenants, Postgres RLS silently filters rows. For silo tenants, the connection targets a dedicated cluster. Cache keys always include tenant_id as a prefix to prevent cross-tenant cache collisions.

Async path: Usage events flow from the application layer into Kafka, partitioned by tenant_id. ClickHouse consumers deduplicate by event_id and append to tenant-partitioned tables. A nightly reconciler aggregates usage and pushes meter events to Stripe with idempotency keys^[5:1].

Deep Dives#

Tenant isolation: pool, bridge, and silo in one platform#

The isolation model is the single decision that determines cost structure, blast radius, compliance posture, and operational complexity. AWS SaaS Lens formalizes three models^[1:1][10]:

Pool (shared DB + RLS): All tenants share tables. A tenant_id column on every row plus Postgres RLS policies enforce isolation at the database layer^[8:1][9:1]. The application sets SET LOCAL app.tenant_id = '<uuid>' at connection checkout. RLS evaluates the USING clause per row before any user predicate; rows that fail are silently filtered. FORCE ROW LEVEL SECURITY closes the table-owner bypass loophole^[8:2]. Admin scripts, bulk jobs, and reporting queries all inherit the filter automatically^[8:3].

Bridge (schema-per-tenant): Each tenant gets a dedicated Postgres schema inside a shared cluster. Stronger isolation than RLS (no accidental cross-schema queries), easier per-tenant migrations, but more schema objects to manage^[11][12].

Silo (dedicated cluster): Enterprise tenants get their own database instance, Redis, and optionally their own Kubernetes namespace. Hardest isolation, cleanest cost attribution, required for HIPAA or data-residency compliance^[4:1].

A mature platform runs all three simultaneously. Tier assignment:

Tiers map to isolation models; tenants promote up the spectrum as they grow or sign enterprise contracts.

Promotion mechanics: When a pool tenant's CPU exceeds 30% of the shared cluster or data exceeds 1 TB, the system triggers a migration. The pattern is dual-write to both old and new locations, backfill historical data, verify checksums, cut reads over, then clean up. Shopify's Pod Mover migrates a pod between data centers in under a minute without dropping requests^[3:1].

Noisy-neighbor containment#

In the pool model, any tenant can theoretically consume 100% of shared resources. Defenses are layered:

API rate limits: Per-tenant token buckets in Redis. Free tier: 100 req/sec. Pro: 1,000 req/sec. Enterprise: unlimited (silo absorbs it). Overage returns structured 429s with Retry-After headers.

Connection pool quotas: pgbouncer configured with per-tier pool sizes. Free tenants share a 50-connection pool. Pro tenants get 200 connections. Enterprise tenants connect to their dedicated instance with 500 connections.

Statement timeouts: SET LOCAL statement_timeout = '10s' per tenant context. A runaway query from one tenant cannot hold connections indefinitely.

Governor limits (Salesforce model): Hard caps enforced at the runtime layer before execution. Salesforce limits each transaction to 100 SOQL queries, 50,000 total rows across all SOQL queries per transaction, and 10,000 ms CPU^[13]. Queries estimated to exceed limits are refused before they run, not killed mid-execution.

Cellular architecture (Shopify model): Shopify's pods are fully isolated datastores (MySQL shard, Redis, Memcached) serving a subset of shops. No cross-pod runtime communication. A noisy merchant's load cannot reach across pods^[3:2].

Metering pipeline and exactly-once billing#

Metering is billing. Every usage event becomes a dollar on an invoice. The pipeline must be exactly-once: double-counting overcharges customers; under-counting leaks revenue.

Architecture:

Exactly-once metering via idempotency keys; nightly reconciliation catches drift before it reaches customers.

Idempotency: Stripe's Meter Event API accepts an identifier field per event. If a caller retries on timeout, the duplicate identifier is silently dropped^[5:2]. Without it, Stripe auto-generates one, but the caller cannot dedupe across retries.

Rate limits: Stripe's v1 meter endpoint handles 1,000 events/sec; the v2 stream path handles 10,000 events/sec^[5:3]. For 7K events/sec aggregate, batch and fan out across multiple Stripe customer meter streams.

Reconciliation: A nightly job compares ClickHouse aggregates against Stripe-recorded usage per tenant. Drift beyond 0.01% triggers an alert. At $100M ARR, 0.01% is $10K/month, the threshold where customers notice and dispute.

Timestamp tolerance: Stripe accepts events with timestamps within the past 35 days and no more than 5 minutes in the future^[5:4]. This accommodates clock drift and late-arriving events from mobile clients.

Tenant onboarding as a state machine#

Provisioning is modeled as an explicit, idempotent, checkpointable state machine. Each step has an idempotency key; failures retry from the last successful checkpoint.

The 8-step provisioning state machine; every transition is idempotent and retryable from the last checkpoint.

Pool tenants provision in seconds (a metadata insert plus config rows). Silo tenants provision in minutes to hours (spin up a database, configure backups, set up DNS). This asymmetry is why enterprise tenants sign contracts before provisioning and free tenants self-serve into the pool.

Real-World Example#

Shopify Pods: cellular multi-tenancy for millions of stores.

Shopify's journey illustrates the full isolation spectrum. Until 2015, Shopify ran on a single MySQL database. In 2015, they sharded by shop ID. But sharding alone left the application brittle: any Sharding.with_each_shard code fanned out to all shards and failed if any one was down^[3:3].

In 2016, Shopify reorganized into pods. A pod is a fully isolated set of datastores (MySQL shard, Redis, Memcached) serving a subset of shops. Stateless workers can talk to any pod, but each unit of work (request or job) is pinned to exactly one pod via the Sorting Hat load balancer, which matches each request to a pod using rules and injects a header^[3:4].

The key insight: no cross-pod runtime communication. A failure in Pod 7 cannot cascade to Pod 12. Each pod has a paired data center for DR. Pod Mover migrates a pod between data centers in under a minute without dropping requests or jobs^[3:5].

For their largest merchants (Shopify Plus "whales"), Shopify provides premium isolation: effectively their own pod or a lightly populated one. This is the silo model dressed in cellular clothing.

The contrast with Salesforce is instructive. Salesforce runs hundreds of thousands of customer organizations on a single shared multitenant database with a single schema, using metadata-driven virtual schemas and governor limits^[2:1]. Salesforce chose radical pooling with runtime enforcement. Shopify chose radical isolation with cellular boundaries. Both work at massive scale. The difference is the failure mode: Salesforce's governor limits prevent noisy neighbors but create a complex runtime; Shopify's pods prevent cross-tenant interference structurally but require pod-aware tooling for every operation.

Trade-offs#

The biggest meta-decision: pool vs. silo is not a one-time choice. Real SaaS runs all three simultaneously, assigning tenants by tier. The architecture must support promotion (pool to silo) without downtime. Design the migration path before the first whale forces it at 2 a.m.

Scaling and Failure Modes#

At 10x (500K tenants, 500K req/sec):

• Tenant router cache becomes the bottleneck. Mitigation: Atlassian's approach with in-process sidecar caches achieving >99.5% hit ratio and 11 microsecond p50 latency^[7:1]. Background refresh prevents cold-cache storms.
• Pool DB connection limits saturate. Mitigation: promote heavy tenants to bridge; add read replicas for pool.

At 100x (5M tenants, 5M req/sec):

• Single-region architecture fails availability targets. Mitigation: cell-per-AZ architecture (Slack model). Each AZ runs a full stack; edge Envoys drain a failing AZ by reweighting in seconds^[14].
• Metering pipeline Kafka partitions become hot. Mitigation: re-partition by (tenant_id, event_type) for better distribution; increase partition count.

At 1000x:

• Architectural rewrite: move to Shopify-style pods where each pod is a self-contained unit. Global control plane for tenant CRUD; data plane fully isolated per pod.

Failure modes:

• Cross-tenant data leak: Immediate P0 incident. Response: revoke affected tenant's sessions, audit all queries in the window, notify the supervisory authority within 72 hours per GDPR Article 33, and notify affected data subjects without undue delay per Article 34^[15]. Prevention: RLS at DB layer, not application trust^[9:3].
• Tenant router cache poisoned: Atlassian's 2019 TCS incident: a data migration brought the service up with empty tables; sidecars cached the empty state and products went offline across regions^[7:2]. Mitigation: anomaly detection on 200/404 response ratios; dummy "content-check" keys fetched periodically.
• Billing pipeline drift: Stripe-recorded usage diverges from internal totals. Response: halt invoicing, run reconciliation, issue credits. Prevention: idempotency keys on every meter event^[5:5].

Common Pitfalls#

Follow-up Questions#

Exercise#

Exercise 1: Design the noisy-neighbor detection and response#

Your platform runs 5,000 tenants in a shared Postgres pool. One tenant's analytics dashboard query is scanning 50M rows and holding connections for 30+ seconds, causing p99 latency to spike from 200 ms to 8 seconds for all other tenants. Design the detection, immediate response, and long-term prevention.

Key Takeaways#

• Tenancy is a spectrum, not a switch. Pool is cheap and risky; silo is safe and expensive. Run all three simultaneously and price the isolation level.
• tenant_id propagation is a safety property. Enforce it at the database layer via RLS, not in application code. Test cross-tenant reads fail closed.
• Metering is billing. Exactly-once semantics with idempotency keys and monthly reconciliation are non-negotiable at SaaS scale^[5:7].
• Noisy-neighbor containment requires layered defenses. Rate limits, connection quotas, statement timeouts, and governor limits together, not any one alone.
• Design promotion paths before the first whale. Pool-to-silo migration at 2 a.m. under pressure is how you lose your largest customer.
• Crypto-shredding solves GDPR for append-only stores. Destroy the key, orphan the ciphertext^[16:2].

Further Reading#

• AWS SaaS Tenant Isolation Strategies. The canonical silo/bridge/pool taxonomy with concrete implementation patterns for each model.
• Shopify: A Pods Architecture To Allow Shopify To Scale. The original pods post explaining cellular isolation, Sorting Hat routing, and Pod Mover migration.
• Atlassian: Six-Nines Availability via Tenant Context Service. The deepest public writeup of a tenant-metadata service at scale, including poisoned-cache detection and recovery.
• PostgreSQL Row Security Policies. The reference for RLS: USING vs WITH CHECK, FORCE ROW LEVEL SECURITY, and interaction with the query planner.
• Stripe: Record Usage with the API. Primary source on idempotent meter events, rate limits, and timestamp tolerance windows.
• Slack: Migration to a Cellular Architecture. Why AZ became the cell boundary and how Envoy xDS enables seconds-order traffic draining.
• Conduktor: Crypto Shredding for Kafka. Practical implementation of GDPR-compliant deletion in append-only streaming systems.
• Salesforce Platform Multitenant Architecture. Metadata-driven design, governor limits, and the flex-column data model at 150K+ orgs.

Flashcards#

References#