Design a URL Shortener: From Requirements to Bottlenecks
A worked, interview-ready design for a TinyURL-style service. Start with the product contract, estimate traffic, storage, and bandwidth, then define APIs, indexed data, collision-safe code generation, cache-backed redirects, and the bottlenecks that determine when to replicate, partition, or migrate storage.
Requirements: Define the Contract Before Choosing Components
Start by fixing the contract, not by naming a database. The core product has two actions: create a mapping from a long URL to a short code, and resolve that code back to the destination. Before drawing the request path, ask whether users may choose custom aliases, set an expiration time, or delete links. Also decide whether shortening the same long URL twice must be idempotent—returning the existing code—or may create two independent codes. These choices change uniqueness, ownership, and cleanup requirements.
Keep product behavior separate from operational targets. For redirects, set an explicit latency and availability objective; a reasonable starting point is p99 under 50 milliseconds end to end and 99.99% availability. Analytics is a separate requirement: collect click events without making the redirect wait for an analytics write. If analytics may lag by seconds, an asynchronous pipeline is acceptable; if every click must be counted synchronously, the redirect path has a different failure and latency budget.
Use this grid to separate the product contract from the operational assumptions that drive the design.
Design a platform-scale distributed rate limiter55 min
Critique: find the failure modes in a given design45 min
Functional
Non-functional
Shorten a long URL into a short one.
Low latency for redirects.
Redirect a short URL to the original long URL.
High availability for redirects.
Support optional custom aliases.
Collect click analytics without making redirects wait on analytics writes.
Support optional expiration time.
Assume 100 M shortens/day, 1 B redirects/day, and a 10:1 read/write ratio.
Decide whether deleting a short URL is supported.
Keep the redirect path read-heavy relative to the shorten path.
Decide whether shortening the same long URL must be idempotent.
The scale assumption is the constraint behind every later component choice. Ask how many URLs are shortened per day and how often those codes are read. Use approximately 100 million shortens per day and a read/write ratio of at least 10:1 as the baseline. A concrete interview assumption is 100 million daily shortens and 1 billion daily redirects: that is roughly 1,200 sustained writes/s and 12,000 sustained reads/s; with a 2× spike factor, that is roughly 2,400 peak writes/s and 24,000 peak reads/s. The read-heavy shape makes redirect availability and latency more important than optimizing the creation path first.
CHECK YOUR UNDERSTANDING
Which questions must you answer before choosing Redis, a database, or a short-code algorithm?
SHOW ANSWERHIDE ANSWER
Clarify the create and redirect behaviors, custom aliases, expiration, deletion, and idempotent shortening. Then establish redirect latency and availability targets, analytics requirements, retention expectations, and the daily shorten volume plus read/write ratio. Those answers define the workload and the contract that the components must satisfy.
Estimation: Turn Product Assumptions Into Capacity
Before choosing a database or cache, turn the product assumptions into rates. Assume 100 M shortens per day and 1 B redirects per day. Redirects therefore outnumber shortens by 10:1, so the redirect path—not the write path—sets the first throughput target.
Traffic
Shortens: 100,000,000 URLs/day ÷ 86,400 seconds/day = 1,157 writes/s, which rounds to approximately 1,200 sustained writes/s. With the required 2× peak factor: 1,200 × 2 = 2,400 peak writes/s—twice the sustained rate.
Redirects: 1,000,000,000 redirects/day ÷ 86,400 seconds/day = 11,574 reads/s, which rounds to approximately 12,000 sustained reads/s. At the same 2× peak factor: 12,000 × 2 = 24,000 peak reads/s—twice the sustained rate.
The resulting peak mix is approximately 24,000 reads/s versus 2,400 writes/s, still a 10:1 read-to-write ratio. Your redirect service and its lookup tier must therefore absorb roughly ten times as many operations as the shortening path.
Storage
Rows: 100,000,000 rows/day × 365 days/year × 5 years = 182,500,000,000 rows, or approximately 180 B rows over five years.
Raw storage: 182,500,000,000 rows × 500 bytes/row = 91,250,000,000,000 bytes, or approximately 90 TB raw. The 500-byte estimate includes the seven-character short code, a long URL of up to 2 KB, timestamps, and a user ID; the row count is the larger multiplier by far.
This is raw storage, before replication, indexes, and backups. The important comparison is scale: you are retaining roughly 180 B rows and approximately 90 TB, not merely serving 24,000 peak lookups per second. That storage footprint forces an explicit decision about whether a relational primary with read replicas remains appropriate or whether you need a distributed key-value or wide-column store.
Redirect bandwidth and cache
Redirect bandwidth: 12,000 redirects/s × 200 bytes/response = 2,400,000 bytes/s, or approximately 2.4 MB/s outbound. Each request contributes only about 200 bytes, so the payload is small relative to the 12,000 requests/s it accompanies: this is primarily a latency and request-throughput problem, not a bandwidth problem.
Hot cache: if the top 20% of URLs produce 80% of traffic, caching the assumed 20 M hot codes at 500 bytes/code requires 20,000,000 × 500 = 10,000,000,000 bytes, or approximately 10 GB. That assumed hot set is about 10 GB ÷ 90 TB = 1/9,000, roughly 0.01% of the five-year raw database footprint, while covering 80% of requests under the stated concentration assumption.
API Design: Keep the Surface Small and Make Redirect Semantics Explicit
Two endpoints, two jobs
Keep the public contract small: one operation creates a mapping, and one operation consumes it. POST /shorten is the right shape for creation because the service allocates the short code and persists a new relationship. The request accepts the required long_url plus optional ttl and alias; the response returns the allocated short_code. A custom alias still goes through the same creation path, so it must compete for the same short-code namespace and uniqueness rules.
http
POST /shorten
Content-Type: application/json
{
"long_url": "<long URL>",
"ttl": "<optional expiration time>",
"alias": "<optional custom alias>"
}
Response
Content-Type: application/json
{
"short_code": "<short code>"
}
GET /<short code>
HTTP/1.1 302 Found
Location: <long URL>
# Redirect choice: use 302 when analytics are needed; consider 301 when the destination is permanent and analytics are not needed.
# After returning the redirect, fire an async analytics event.
The interface separates shortening from redirecting and makes the analytics trade-off explicit.
The redirect endpoint is a GET because the client supplies the short code and expects the service to resolve it, not create another resource. The response's Location header carries the destination. Use 302 Found when the service must observe redirects: the request reaches the redirect service, which can emit a click event asynchronously without making the user wait for analytics storage. That keeps click recording off the latency-critical response path.
This choice is a product decision, not a cosmetic status-code change. If a fraud-detection pipeline must observe every click, return 302 and publish the click event to an asynchronous downstream pipeline after issuing the redirect. A 301 may make repeat visits bypass the service entirely, so no downstream consumer can infer that every click was observed. If analytics are not required and the destination is genuinely permanent, 301 can reduce repeat traffic through the service, but it also gives up that control.
CHECK YOUR UNDERSTANDING
A fraud-detection pipeline must observe every click on a short URL. Should the service use 301 or 302, and how should it expose the click event downstream?
SHOW ANSWERHIDE ANSWER
Use 302 so redirects continue passing through the service. Return the Location header promptly, then emit a click event to an asynchronous analytics pipeline; do not block the redirect on writing the analytics record. A 301 can be cached by browsers, causing later clicks to bypass the service and making “every click” unobservable.
Data Model: Make the Short-Code Lookup the Fast Path
The mapping table should serve the redirect path first: given one short_code, return one destination without scanning unrelated mappings. Keep the record deliberately narrow: a roughly 7-character code, a long URL up to 2 KB, creation and expiry timestamps, and the creating user's ID. The short code is the identity of the mapping; the long URL is an alternate lookup only when the product promises idempotent shortening.
Define a mapping record keyed by a unique short_code, with the long URL, creation and expiry timestamps, and the creating user's ID. If idempotent shortening is required, enforce uniqueness for long_url in the chosen database; otherwise do not add that uniqueness constraint.
The unique short_code key makes the redirect lookup a single-column point read rather than a scan of unrelated mappings. A uniqueness constraint for long_url is optional from a product perspective: keep it when the same long URL must return the same code, and omit it when every shorten request may create a new mapping. Store creation and expiry timestamps for lifecycle checks, and store the creating user's ID when ownership is in scope.
A short-code index resolves the redirect point read directly, while the long-URL index is an optional idempotency path.
Follow the upper lookup path from one short code through its index to one mapping row; the separate long-URL path is the additional work required for idempotent shortening.
Uniqueness and idempotent shortening
Generated codes and custom aliases must share the same uniqueness boundary. Store both in short_code; otherwise an alias can be accepted even though it collides with a generated code, and the redirect path has no unambiguous row to return. When idempotent shortening is required, look up the existing mapping by long URL and enforce uniqueness when creating the mapping. If two requests race, the uniqueness constraint—not an application-level check alone—must decide which mapping wins; the losing request retries its lookup and returns the committed mapping.
Store and partition choice
At the stated retention horizon, 100 million mappings per day over five years is about 180 billion rows; at approximately 500 bytes per row, that is about 90 TB of raw mapping data. Pair that footprint with roughly 12,000 redirect reads per second sustained and roughly 24,000 at the stated 2× peak. PostgreSQL with a unique indexed short_code and read replicas is a defensible starting point when its operational limits and replication lag are acceptable. A key-value or wide-column store becomes attractive when the point-read workload and dataset no longer fit the chosen relational topology; the numbers justify the comparison, not an automatic migration.
Do not shard merely because the table is large. Begin with one primary and replicas if that topology can absorb the write rate, index maintenance, storage, and recovery requirements. When one primary, its indexes, or replica capacity becomes the limiting resource, consider partitioning by short_code so a redirect point read can be routed by its lookup key. A query organized by created_at cannot be answered from one such partition and requires a fan-out across partitions. The choice therefore favors the dominant access pattern over convenient operational reports.
CHECK YOUR UNDERSTANDING
A customer requires the same long URL to always produce the same short code. What lookup or index do you add, and what does it change on the write path at scale?
SHOW ANSWERHIDE ANSWER
Add a unique index or uniqueness constraint on long_url, alongside the primary key or unique index on short_code. The shorten path first performs the long-URL lookup and returns the existing mapping when found; concurrent creators rely on the uniqueness constraint to coordinate, then the losing request reads the committed row. This adds an indexed lookup and contention/maintenance to every idempotent shorten, so at roughly 1,000 sustained writes per second and roughly 2,000 writes per second at a 2× peak, you must include that work when sizing the write store.
High-Level Design: Separate the Write Path From the Redirect Path
The design has two request paths with different bottlenecks. Shortening is a write: validate the request, allocate a code, and persist the mapping. Redirecting is a high-volume point read: resolve the code with the lowest possible latency. Put both behind a load balancer, but keep the application workers stateless so you can add or remove workers without moving user state.
On the shorten path, the client sends POST /shorten to the load balancer. A stateless shorten worker generates the code and writes the mapping to the database primary. After the primary confirms the write, the worker can pre-warm Redis with the new mapping; lazy population is also valid when the link is unlikely to be used immediately. The database primary owns writes so code uniqueness and expiry metadata have one authoritative write path.
On the redirect path, the client sends GET /{short_code} through the same load balancer to a stateless redirect worker. The worker checks Redis first. A cache hit returns the destination without touching the database. On a miss, it reads from a database replica, stores the result in Redis, and returns the redirect. Replicas keep the roughly 12,000 sustained reads per second away from the primary while the cache absorbs repeated requests; the estimated 90 TB raw database is also far larger than the roughly 10 GB hot working set, so caching avoids treating the whole corpus as a hot read workload.
Writes use the primary; redirects use Redis first and replicas only on cache misses.
Follow the two lanes from the load balancer: writes terminate at the database primary, while redirects take the Redis-hit path or fall back to read replicas.
The estimates determine how far you need to go. A read-heavy workload—about 1 billion redirects per day versus 100 million shortens, or roughly a 10:1 read-to-write ratio—justifies Redis and read replicas before database sharding. Start by measuring whether an indexed PostgreSQL primary with replicas meets the latency and throughput target. If the 90 TB, high-read design exceeds the storage, replication, or operational limits of that choice, evaluate a key-value or wide-column store and explain which limit it removes. The decision should follow the measured workload, not the database brand.
CHECK YOUR UNDERSTANDING
Why should the redirect service read replicas instead of sending every cache miss to the database primary?
SHOW ANSWERHIDE ANSWER
The redirect path has roughly ten times as many requests as the shorten path. Replicas can serve the read-heavy fallback workload while the primary remains responsible for writes, uniqueness, and authoritative mapping changes. Redis handles hits; replicas handle misses, so neither every redirect nor every read fallback concentrates on the primary.
Short-Code Generation: Uniqueness Without a Global Write Hotspot
Short-code generation has two separate requirements: the code must be compact enough for a URL, and its uniqueness must survive concurrent writes from many application servers. Two design families are useful: derive the code from the long URL, or allocate a unique integer and encode it.
Hashing: compact, independent, but collision-prone
A hash-based design canonicalizes the long URL, computes an MD5 or SHA-256 digest, and encodes a truncated result with base62. With seven characters, the code space is 62^7, approximately 3.5 trillion combinations. That is a large space, but truncation means two different URLs can still produce the same seven-character code; a hash is not a uniqueness guarantee.
The write must therefore make the short code unique at the database boundary, not only in application memory. Insert the mapping under a unique constraint on short_code. If the insert reports a conflict, look up the existing mapping: return it when it belongs to the same long URL, preserving idempotent shorten behavior; otherwise derive a new candidate, such as by adding a salt and hashing again, then retry. A suffix-and-rehash strategy is another valid collision-resolution path, but every retry must still pass through the same uniqueness check.
text
HASH CREATION
create_short_url(long_url):
candidate = base62(truncate(MD5(long_url)))[0:7]
loop:
result = INSERT mapping(long_url, short_code = candidate)
if result succeeds:
return candidate
if result is a UNIQUE constraint violation:
existing = find mapping by short_code = candidate
if existing.long_url == long_url:
return existing.short_code // idempotent mapping
candidate = base62(truncate(SHA-256(long_url + salt)))[0:7]
retry
COUNTER-RANGE ALLOCATION
server:
if local_range is exhausted:
local_range = request the next range of 1 M IDs
id = next unused ID in local_range
local_range advances
short_code = base62(id)
persist mapping(short_code, long_url)
if server crashes before consuming local_range:
the unconsumed IDs become a lost-range gap
request another range after restart
The hash path retries after a uniqueness conflict, while range allocation generates codes locally and can leave a gap after a crash.
The important failure distinction is between a duplicate request and a true collision. The former should return the existing mapping instead of creating another code. The latter should never overwrite the existing owner of the code. This turns a probabilistic generator into a safe write path, at the cost of an extra lookup and retry only when the candidate is already occupied.
Counters: uniqueness by construction, coordination by design
A global atomic counter takes the opposite approach. A counter service assigns the next integer, and the application base62-encodes it. Redis's INCR is one example of the allocation operation. This is fast and collision-free, but every shorten request depends on the counter service: it is a write bottleneck and a single point of failure. Sequential IDs also expose creation order and make enumeration easier.
Range allocation removes that central operation from every request. A ticket service hands each application server a disjoint range, such as 1 M IDs. The server consumes IDs locally, base62-encodes each one, and requests another range only after exhausting its current range. If it crashes midway, the unused IDs in that range become a gap. That gap is acceptable: uniqueness matters, not contiguous numbering. Base62 makes the allocated value compact, but it does not hide its sequential nature.
This is the practical compromise for many designs: the allocator coordinates ranges rather than individual writes, while application servers generate codes locally without collisions. Custom aliases must enter the same uniqueness boundary. Store custom aliases under the same unique index or namespace as generated codes, and reject conflicts. Without that check, users can squat on codes or race a generated mapping for the same visible path.
Hash generation resolves an occupied code with a guarded retry, while range allocation trades contiguous IDs for distributed uniqueness.
The left side shows two URLs converging on one truncated hash before a unique-constraint retry, while the right side shows disjoint 1 M-ID ranges and the unused fragment left by a crashed server.
CHECK YOUR UNDERSTANDING
Two different long URLs produce the same seven-character code from your truncated hash. What exact write path detects and resolves the collision?
SHOW ANSWERHIDE ANSWER
Compute the candidate and attempt the insert guarded by the unique short_code constraint. On a uniqueness violation, look up the existing mapping. If its long URL matches, return the existing short code for idempotent behavior. If it differs, generate a new candidate—such as base62(MD5(long_url + salt))[0:7]—and retry until the insert succeeds.
Caching and Redirect Performance: Protect the Database on the Hot Path
Cache-aside on the redirect path
Treat the short-code mapping as the cache value: the code is the key, and the long URL plus its expiry and deletion state are the value. A redirect worker checks Redis first. On a hit, it returns the redirect without consulting the database. On a miss, it reads the mapping from the database, stores that mapping in Redis with an expiry aligned to the URL's own expiry, and returns the redirect. This keeps the database off the normal read path while preserving it as the source of truth.
Permanent URLs can use long cache lifetimes, but a permanent cache entry is safe only while the destination and deletion state cannot change. If either can change, active invalidation must remove or replace the cached mapping when the write occurs. Expiring URLs need a cache lifetime that does not outlive the mapping; otherwise Redis can redirect to a URL that the product considers expired. LRU eviction is a natural fit for the remaining capacity: codes that stop receiving traffic become cold and age out, preserving space for links that are being clicked.
Population and the viral-link failure mode
Lazy population is the simpler write policy: shortening writes the mapping to the database, and the first redirect fills Redis. Write-through or explicit pre-warming also writes the new mapping to Redis during shortening. That adds write-path coordination, but it removes the first-hit penalty for a link you already expect to be popular. For an ordinary link, lazy population avoids work that may never be needed; for a campaign link, pre-warming can be justified by the product context.
Consider a marketing link created 10 seconds before a television advertisement airs. The first viewers arrive together while the cache is cold. Without coordination, thousands of workers observe the same miss and issue the same database read. The database receives a burst for one value even though one read could have populated the entire hot entry. Protect the refill with a per-key mutex or semaphore: one worker owns the refill, while the others wait briefly or retry the cache. Probabilistic early expiry can spread refills before the entry expires, reducing the chance that many requests discover an empty key at once.
Request coalescing turns a burst of concurrent misses into one database read and one cache refill.
Follow the single refill from the Redis restart and concurrent client misses to one database read and the cache becoming hot.
Cold start after a Redis restart
A Redis restart removes the working set, so the system must treat recovery as a controlled warm-up rather than allowing every miss through. Request coalescing still permits only one refill for a given short code. Replicas provide additional database capacity for fallback reads, and warm-up should prioritize known hot codes instead of replaying the entire keyspace. During recovery, the redirect service should keep serving cache hits, route misses through the refill guard, and apply its fallback behavior when the database or cache is unavailable rather than multiplying retries against the primary.
The checkpoint scenario makes the scale problem explicit: 50,000 redirects per second after a cache loss is roughly four times the 12,000-read-per-second sustained estimate. If all 50,000 requests independently miss, the database sees the burst; if requests for the same viral code coalesce, one read repopulates the entry and the remaining requests wait or retry. After the cache is hot, the redirect path returns to serving the repeated mapping from memory. The operational questions are whether the hit rate recovers, how long warm-up takes, and whether database fallback remains bounded during that interval.
CHECK YOUR UNDERSTANDING
Redis has just restarted and lost its data while a viral link is receiving 50,000 redirects per second. What happens, and how do you prevent the database from falling over?
SHOW ANSWERHIDE ANSWER
The redirect workers see misses, but they do not all query the database. A per-key mutex or semaphore lets one worker refill the viral code while the others wait or retry. Controlled warm-up prioritizes known hot codes, replicas serve bounded fallback reads, and the service avoids multiplying retries against the primary. Once the single database read repopulates Redis, subsequent redirects are cache hits. Monitor hit-rate recovery, miss latency, and fallback load while the cache is cold.
Bottlenecks and Scaling: Identify the Next Limit Before You Add a New Tier
Scale the system by the constraint that is actually closest to failure, not by adding the most sophisticated storage tier. At the estimated rate, redirects are roughly 10× writes: about 12,000 reads/s versus 1,200 writes/s sustained, with peak planning roughly doubling both. Redis and read replicas therefore absorb redirect pressure; the single database primary is the first write-side constraint as shorten traffic rises.
A shorten service reaching 5,000 writes/s on one PostgreSQL primary is handling roughly four times the estimated sustained write rate. Before changing databases, reduce contention where the write path permits it: batch writes, scale the primary vertically, and let application servers pre-fetch ID ranges instead of requesting every identifier from one global counter. Range allocation costs some operational complexity and can strand unused IDs when a server crashes, but it removes per-request counter contention.
Use the first mitigation to relieve the current constraint; reserve structural migrations for sustained growth or a failure mode the incremental fix cannot address.
Bottleneck
Observable symptom
First mitigation
Structural migration
Database-primary write saturation
At 5,000 writes/s on one PostgreSQL primary, shorten requests queue or fail while Redis and read replicas continue absorbing redirect reads.
Batch writes where correctness permits, scale the primary, and use range allocation so application servers pre-fetch ID ranges instead of contending on one global counter.
Partition or shard the mapping store; move ID generation to a counter service when the primary or global counter remains the write bottleneck. A single global auto-increment counter can become a write bottleneck and a centralized dependency as the system is distributed.
Cache loss or hot-key overload
After a Redis restart, a viral link produces a cache miss for every redirect; a hot key can concentrate traffic and create a cache stampede against the database.
Use cache-aside with lazy repopulation, protect misses with a mutex or semaphore, and pre-warm especially hot links on the write path.
Add cache capacity or a distributed cache tier; keep the database path protected while the cache repopulates.
Read-replica lag
Redirect reads routed to a lagging replica can return stale mappings or miss a newly created short URL; the primary receives more reads when replicas cannot keep up.
Keep the cache in front of the database, route cache misses according to freshness needs, and use the primary when read-after-write behavior is required.
Add replicas or move the read-heavy mapping path toward a KV or wide-column store; retain the primary for writes and authoritative reads.
Storage/partition growth
The approximately 90 TB five-year footprint and high read QPS make one unpartitioned database harder to operate even when Redis absorbs most repeat reads.
Keep the short-code lookup indexed, add read replicas, and measure partition size, replica lag, and cache effectiveness before changing storage technology.
Partition or shard by short key or another access-aligned key; if the footprint and read budget continue to grow, migrate incrementally toward a KV or wide-column store.
The decision grid orders the first response before the structural migration. The important escalation is from an indexed PostgreSQL primary with replicas, to partitioning or sharding when the footprint and write budget justify it, and only then toward a KV or wide-column store when the read-heavy mapping path keeps outgrowing the relational setup. The five-year estimate is approximately 90 TB: 180 billion rows at about 500 bytes each. That is not a reason to abandon PostgreSQL immediately, but it is a reason to measure partition size, replica lag, and cache effectiveness from the start.
A single global auto-increment counter becomes shard-hostile because every shard still depends on one allocation point. It also exposes creation order. If multiple database nodes are required, partitioning or sharding the mapping store removes the primary's storage and write ceiling, but adds routing, rebalancing, and failure-recovery work. Moving ID generation to a counter service removes identifier allocation from the database, but creates another highly available dependency; range allocation is the less centralised intermediate step.
The remaining limits are consistency and concentration rather than raw component count. Replica lag can make a newly created code invisible on a redirect miss, so route freshness-sensitive misses to the primary while keeping ordinary reads behind the cache. Partitioning by an access-aligned key or migrating incrementally to a KV or wide-column store reduces pressure, but each step increases operational complexity and migration risk. The design's unavoidable failure is a simultaneous cache loss during a viral event: without stampede protection and enough authoritative capacity, no database choice makes that burst free.
CHECK YOUR UNDERSTANDING
Your shorten service reaches 5,000 writes/s on one PostgreSQL primary. What do you change first, and when do you shard or move ID generation?
SHOW ANSWERHIDE ANSWER
First measure the primary's write queue and failure rate, then batch writes where correctness permits, scale the primary, and use range allocation so application servers do not contend on one global counter. If the primary remains the write bottleneck, partition or shard the mapping store. Move ID generation to a counter service when identifier allocation itself remains the limiting point; that removes database contention but adds a new highly available dependency.
KEY TAKEAWAYS
Size the redirect and shorten paths separately: 100 million shortens/day produces roughly 1,200 sustained writes/s, while 1 billion redirects/day produces roughly 12,000 sustained reads/s.
A unique short-code boundary must protect both generated codes and custom aliases; hash-based generation still needs a uniqueness check and retry path.
Use Redis as a cache-aside layer for redirect lookups, but protect cache misses with request coalescing so a hot-key burst does not become a database outage.
Choose 301 versus 302 based on whether the service must continue observing clicks and enforcing destination changes or expiry.
Scale the component closest to failure: replicas and caching address read pressure, while batching, range allocation, partitioning, or a storage migration address write and capacity limits.