Designing a Pastebin: From Requirements to Bottlenecks
A worked Pastebin design that turns an interview-sized scope into capacity estimates, an explicit API, a metadata-and-object-storage model, cache-aware request paths, and operational fixes for expiry, deletion, hot keys, and ID collisions.
Requirements and scope
Start by fixing the product boundary. A paste is plain text, capped at 10 MB; binary and rich content remain open scope questions, and the MVP requires no authentication. Users can create, retrieve, and delete pastes through an explicit API Design: the client-server contract for operations, inputs, outputs, and failure cases. That contract must settle expiry and deletion behavior before you choose a database or storage service.
Functional requirements
The MVP accepts anonymous pastes, so there are no accounts or ownership records to authorize a delete. Instead, creation returns a deletion token; possession of that token grants deletion rights. Each paste receives a short identifier and an optional user-chosen TTL from 1 hour to forever. After the expiry time, the paste must not be served: distinguish an expired paste from an identifier that never existed, using the API's gone-versus-not-found contract. The system stores paste content in Object Storage, a flat key-value store for arbitrary blobs: the application writes content under a key and retrieves it by that key, without forcing the blob into a relational row.
Requirements matrix separating the initial scope from decisions that remain open.
Design a platform-scale distributed rate limiter55 min
Critique: find the failure modes in a given design45 min
Requirement
MVP decision
Open follow-up
Content
Plain text only; 10 MB maximum per paste; store content in object storage (S3).
Support binary or rich content, and revisit the size cap.
Identity
Anonymous pastes; no user accounts required. Issue a deletion token at creation time.
Decide whether accounts, ownership, and authenticated private pastes are needed.
Expiry
User-chosen TTL from 1 hour to forever. An expired paste is gone; distinguish expired from never found.
Define the exact response contract and cleanup timing.
Read/write mix
Read-heavy workload; roughly 10:1 read-to-write ratio. Cache popular pastes in memory.
Validate traffic distribution and size the cache for hot pastes.
Consistency
Make creation, deletion, and expiry behavior explicit before choosing components.
Choose strong or eventual consistency for metadata and reads.
Availability
Keep the MVP path simple; decide explicitly how availability and consistency should trade off, and allow asynchronous cleanup.
Set availability targets and define behavior during object-store or metadata-store failures.
Geography
Defer the region choice for the MVP.
Choose single-region or global deployment, including latency and replication requirements.
Scale
About 100M pastes stored; 10 MB maximum paste size; roughly 10:1 read-to-write ratio.
Refine QPS, storage growth, retention, and cache-capacity estimates.
The matrix makes the deliberate omissions visible. In particular, the MVP does not yet decide whether accounts, authenticated ownership, private pastes, a global deployment, or a precise cleanup schedule are required. Those are follow-up decisions, not assumptions hidden inside the design.
Non-functional requirements
Use a working scale of about 100 million stored pastes and a roughly 10:1 read-to-write ratio. Retrieval is the dominant path, with a read-latency target under 200 ms at p99. Make availability versus strong consistency an explicit design decision for ordinary reads; creation, deletion, and expiry semantics must still be explicit when metadata and content are handled by separate systems. The MVP keeps geography open rather than silently assuming either a single region or global replication.
This read-heavy shape makes Caching a first-class requirement, not a later optimization. Caching serves popular pastes from memory, absorbing hot-key traffic and protecting object storage and the metadata database. The cache must respect each paste's expiry rather than applying one lifetime to every entry. The next design step can therefore estimate the 10:1 traffic split, storage growth, identifier space, and the cache footprint before selecting capacity.
Capacity estimation
Traffic and storage
Start with the working assumptions: 10,000,000 new pastes per day, a 10:1 read-to-write ratio, and an average paste size of 10 KB. Convert the daily write volume to seconds: 10,000,000 pastes / 86,400 seconds ≈ 115 writes/second. Reads are ten times the writes: 115 × 10 ≈ 1,150 reads/second. The system is therefore read-heavy: reads outnumber writes by 10:1, so the read path—not paste creation—is the first path to size for.
For storage, multiply the daily paste count by the average content size: 10,000,000 × 10 KB = 100,000,000 KB = 100 GB/day. Across a year, 100 GB/day × 365 days ≈ 35 TB/year at planning precision. That is roughly 365 times the daily growth accumulated over a year, and it rules out treating paste bodies as ordinary relational metadata by default: the database would be carrying tens of terabytes of blob content alongside the small fields needed to find and expire each paste. Put the content in object storage and keep the metadata database focused on identifiers, timestamps, expiry, and storage keys.
ID space and collision handling
A seven-character identifier using a 62-character alphabet has 62^7 ≈ 3.5 trillion possible IDs. Against 1 billion existing pastes, the occupied fraction is 1,000,000,000 / 3,500,000,000,000 ≈ 0.00029, or about 0.03%—roughly one occupied ID in 3,500. A randomly generated candidate must still be checked for uniqueness before it is committed; on collision, generate another candidate and retry. The large space keeps the expected retry rate low without making uniqueness an assumption.
Size the cache for the hot slice
If the top 20% of pastes serve 80% of reads, target that hot slice rather than caching every paste. At 1,150 reads/second, the hot set accounts for 1,150 × 0.80 = 920 reads/second; the remaining pastes account for 1,150 × 0.20 = 230 reads/second. Caching the hot slice can therefore remove about 920 of 1,150 reads/second, leaving 230 backend reads/second—a 5× reduction in storage-facing read traffic. This is the capacity decision: provision the write path for about 115 writes/second, but protect object storage and metadata lookups from the much larger, concentrated read load.
CHECK YOUR UNDERSTANDING
Your short-ID generator chooses random seven-character base-62 strings, and 1 billion pastes already exist. What collision strategy do you use, and how does it affect the write path?
SHOW ANSWERHIDE ANSWER
The space contains about 3.5 trillion IDs, while 1 billion are occupied, so a fresh random candidate lands on an existing ID only about 0.03% of the time—roughly one in 3,500. Still, uniqueness must be enforced atomically: generate a candidate, attempt to reserve it with a uniqueness constraint, and retry generation if the reservation collides. Collisions add occasional retries to the write path; they do not change the read-heavy capacity decision.
API contract
API design is the contract between the client and server: it fixes the request shapes, response meanings, and failure behavior before you choose databases, caches, or object storage. For this MVP, keep the surface small: create a paste, retrieve it by short ID, and delete it with the secret issued during creation.
The contract below makes the creation inputs explicit. content is required; ttl and custom_alias are optional. A successful creation returns both the public short ID and a delete_token, because anonymous clients have no account identity to prove ownership later.
http
POST /pastes
Content-Type: application/json
{"content":"...","ttl":"...","custom_alias":"..."}
HTTP/1.1 201 Created
Content-Type: application/json
{"id":"...","delete_token":"..."}
HTTP/1.1 400 Bad Request
HTTP/1.1 <documented collision status>
GET /pastes/{id}
Accept: text/plain
HTTP/1.1 200 OK
Content-Type: text/plain
...
HTTP/1.1 404 Not Found
HTTP/1.1 410 Gone
DELETE /pastes/{id}
Content-Type: application/json
{"delete_token":"..."}
HTTP/1.1 <documented successful-deletion status>
The contract distinguishes malformed requests, an occupied alias, never-found pastes, expired or deleted pastes, and successful deletion; the API documentation must select the two placeholder status codes.
Use POST for creation because the server allocates the paste ID and creates a new resource. A malformed body—such as missing content or an invalid TTL—gets 400 Bad Request. A syntactically valid request whose requested alias is already occupied receives the documented collision response. The creation response uses 201 Created, while a successful deletion has no response body and uses a status code selected and documented for this API.
Retrieval and expiry
GET /pastes/{id} returns the raw text with Content-Type: text/plain. Return 404 Not Found when the ID has never existed. Return 410 Gone when the paste existed but has expired or was deleted. That distinction gives clients a meaningful lifecycle signal and prevents an expired paste from looking like a spelling mistake.
Expiry is enforced at request time, not only by background cleanup. Once expires_at has passed, the server must return 410 and must not fetch, cache, or otherwise repopulate the paste. Deletion follows the same ownership rule: the caller submits the creation-time delete_token; the service verifies it before removing the paste. The token is the anonymous equivalent of an ownership credential, so it must be treated as a secret rather than included in the public ID.
CHECK YOUR UNDERSTANDING
A paste has a one-hour cache TTL but expires in ten minutes. What should a request at minute 11 return?
SHOW ANSWERHIDE ANSWER
It should return 410 Gone, not the cached content. The read path must check expiry before serving and clamp any cache lifetime to the remaining paste lifetime, so the cache cannot outlive the paste.
Metadata and content model
Keep metadata and content on separate paths
Use a lean metadata record for everything you need to find, authorize, expire, or clean up. A Postgres implementation can use id as the primary key, retain the object-store reference in s3_key, and record created_at, nullable expires_at, and size_bytes. custom_alias is optional but must be unique when present. The row deliberately has no TEXT or BYTEA content column.
sql
CREATE TABLE pastes (
id text PRIMARY KEY,
custom_alias text,
s3_key text NOT NULL,
created_at TIMESTAMP NOT NULL,
expires_at TIMESTAMP NULL,
size_bytes bigint NOT NULL,
UNIQUE (custom_alias)
);
DELETE FROM pastes
WHERE expires_at IS NOT NULL AND expires_at < NOW();
The metadata row contains identifiers, timestamps, and size only; paste content is stored under the S3 object key rather than in this table.
The lookup path is a point query by id; that primary key makes retrieval predictable without scanning paste content. Cleanup has a different access pattern: it finds rows where expires_at is non-null and earlier than the current time. Give that column an index or an equivalent expiry-oriented access path, then let a worker process expired rows in batches. A uniqueness constraint on custom_alias also turns alias collision detection into a database-enforced write check rather than a race-prone application convention.
The paste body belongs in S3-style object storage. Treat it as a flat key-to-value namespace: write the arbitrary blob under s3_key, then retrieve that value by key. The database can search lifecycle metadata, but it cannot search inside the opaque content without a separate indexing pipeline. This split lets the metadata store stay small and keeps content scaling independent from metadata queries.
The metadata database indexes lifecycle fields and points to an opaque object whose bytes live in object storage.
Follow the emphasized `id` to `s3_key` relationship: the database stores searchable lifecycle fields, while the object store holds the opaque blob.
Keys, partitioning, and query boundaries
Use a non-sequential seven-character base-62 id, or an equivalent collision-checked identifier, rather than an auto-increment value. Seven base-62 characters provide approximately 3.5 trillion possible keys, while avoiding a URL that reveals creation order and reducing the need for a single sequential allocator. If you choose random generation, enforce uniqueness and retry on collision; the uniqueness constraint makes the final check authoritative.
For Postgres, keep pastes keyed by id and use the expiry access path for cleanup. If the metadata volume later requires horizontal partitioning, hash-partition by id: it keeps point lookups distributed, but makes a global “all rows expiring before this time” query cross partitions. That trade-off is acceptable when cleanup workers process expiry ranges or maintain per-partition work queues. Do not pretend partitioning makes blob search possible—the content remains addressable only through its object key.
CHECK YOUR UNDERSTANDING
Why is `expires_at` a different access path from `id`, and what does hashing by `id` make harder?
SHOW ANSWERHIDE ANSWER
id supports one-paste retrieval, while cleanup needs an ordered set of expired rows. Hashing by id spreads point lookups, but an expiry scan must visit partitions instead of using one globally ordered range.
Components and request paths
Write path
The client sends a create request to the API servers. The API layer validates the paste, chooses or accepts its ID, and coordinates the two durable writes; it does not make Redis part of the creation path because Redis is a read cache, not the storage of record. First, the API server writes the content to object storage under an s3_key. It then writes the metadata row to the database, including id, s3_key, and expires_at. Only after both writes succeed does it return the short ID and deletion token.
That ordering gives reads a discoverable metadata record only after the content exists. If the object-store write succeeds but the database write fails, the API must not report a successful creation; it retries the metadata write or leaves the object for reconciliation. If the object-store write fails, it likewise does not create a usable metadata row. The cleanup worker handles remaining orphaned objects rather than making the client wait for a distributed transaction.
Read path
For GET /pastes/{id}, the API server checks Redis first. A hit returns the cached content only when its expiry has not passed. A miss goes to the metadata database to distinguish an unknown ID from an expired paste and to obtain the object-store key. An unknown ID returns 404; a known paste whose expires_at has passed returns 410. For an active paste, the API fetches the content from object storage, returns it, and populates Redis with a TTL clamped to the time remaining until expiry.
This places Redis directly in front of object-store reads, so repeated requests for a popular paste do not repeatedly consume object-store or metadata-database capacity. Redis is a disposable read cache rather than the source of record; define an explicit fallback policy for cache failure. If the metadata database is unavailable on a cache miss, the API cannot safely determine whether the ID is missing or expired; if object storage is unavailable, it cannot return an uncached body. Both cases should fail explicitly rather than manufacture a response. A cached, unexpired hit can still be served without contacting those dependencies.
Creation persists content and metadata; retrieval treats Redis as a read-side cache in front of object storage.
Trace POST from the client through the API server to object storage and the metadata database, then trace GET through Redis first and through object storage only after a cache miss.
CHECK YOUR UNDERSTANDING
How do you handle deletion when content lives in object storage and metadata lives in a separate database?
SHOW ANSWERHIDE ANSWER
Treat deletion as coordination between the metadata record and the object. Mark the metadata as deleted or expired so new reads stop using it, then remove the object asynchronously; retries and reconciliation handle either side succeeding before the other.
Object storage and cleanup semantics
The storage boundary is a correctness and operations decision, not just a capacity decision. Keep paste content in object storage and keep the metadata database responsible for identity and lifecycle state. Object storage addresses arbitrary blobs through a flat namespace, so the database needs only short metadata such as id, s3_key, created_at, expires_at, and size_bytes. That lets content capacity and metadata capacity scale independently; it also leaves an attachment point for a CDN when popular, immutable objects need to be served closer to users.
Putting a 5 MB paste in a Postgres BYTEA or TEXT column couples every content read and write to the relational storage path. Large values displace useful metadata and indexes from the buffer pool, increase the data that replicas must copy, and enlarge point-in-time restore work. The problem is not that a relational database cannot store a blob; it is that millions of blobs make the database carry content, replication, backup, and metadata workloads together. Object storage is designed for unstructured objects and scale-out placement. Its durability comes from replication or erasure coding across failure domains, while its API-oriented access model and CDN integration trade some database-style queryability for cheaper, independently scalable blob storage.
Expiry therefore uses an ordered two-phase cleanup. First, a worker changes the metadata row to an expired state, or records that expires_at has passed. Only after that durable database transition does it request deletion of the corresponding S3 object. A GET for an ID that never had metadata returns 404; a GET whose metadata proves expiry returns 410 Gone. The distinction gives clients and operators different information without requiring the object to disappear at the exact expiry instant.
If the database is marked expired but S3 deletion fails or is delayed, subsequent reads still return 410, and a later cleanup attempt can reclaim the object. If S3 deletion succeeds before the database transition finishes—because a worker raced the normal ordering—the metadata row may still describe an active paste whose content is missing. The GET path must not treat that missing object as a new paste: it should fail closed, alert or enqueue reconciliation, and preserve the ID's existing metadata until the lifecycle worker resolves the inconsistency. Make object-deletion retries safe and keep the identifier reserved until metadata and cleanup reconciliation are complete.
This design accepts eventual physical cleanup in exchange for a simple visibility guarantee: once the database says expired, no read path serves the content. A sweeper can find expired metadata, enqueue purge work, and retry failures independently of user traffic. The same metadata key should identify the S3 object throughout its lifetime; reusing an ID while an old object or delayed cleanup still exists would turn an asynchronous failure into possible cross-paste content exposure.
Database expiry becomes visible before object reclamation, and both failure branches converge through retry or reconciliation.
Follow the time axis from expiry marking to asynchronous purge, then compare the two failure branches and the GET behavior that prevents expired or missing content from being served.
CHECK YOUR UNDERSTANDING
Content lives in S3 while expiry state lives in the database. How do you handle deletion, and what happens if the S3 delete succeeds but the database update does not—or if the database update succeeds but S3 deletion fails?
SHOW ANSWERHIDE ANSWER
Make the database lifecycle transition authoritative and perform it first: mark the paste expired, then enqueue an idempotent S3 deletion with retries. If the database update succeeds but S3 deletion fails, GET checks the expired metadata and returns 410 Gone; the object remains temporarily but is not served, and a later retry removes it. If S3 deletion happens first and the database update fails, the metadata can still look active while its object is missing. GET must fail closed rather than interpret that as a new paste, and reconciliation should complete the metadata transition or raise an operational alert. A never-existing ID remains 404 Not Found.
Expiry-aware caching for hot pastes
A paste retrieval path should be cache-aside: the application decides when to read from Redis, when to consult metadata, and when to fetch the blob from object storage. That fits a read-heavy service because cold content is loaded only when requested, while popular content stays close to the API servers. It also avoids making every paste creation pay the cost of populating a cache entry that may never be read.
Write-through is not the natural fit here. It sends each write to both the cache and the database simultaneously, which keeps cached values fresher but adds write latency and creates a failure case when one write succeeds and the other fails. Paste creation should establish the durable metadata and content first; retrieval can populate the cache on demand.
text
GET /pastes/{id}
value = L1[id]
if value exists and value.expires_at > now:
return value.content
value = GET short_key
if value exists and value.expires_at > now:
return value.content
metadata = fetch metadata for id
if metadata does not exist:
return 404
if metadata.expires_at is not null and metadata.expires_at <= now:
return 410
if another request is loading id:
wait for that request
retry
content = fetch content from S3
if metadata.expires_at is null:
cache_ttl = configured cache policy
else:
cache_ttl = min(metadata.expires_at - now, max_cache_ttl)
store content in Redis with cache_ttl
return content
The read path handles never-expiring pastes through the configured cache policy and applies an expiry-aware TTL before storing a fetched value.
The important ordering is the expiry check. A hit is usable only while its stored expiry is later than the current time; otherwise the request continues through the durable path. On a miss, metadata distinguishes a never-existing paste from an expired one: return 404 for the former and 410 for the latter. For a live paste, clamp the cache lifetime to the smaller of the paste's remaining lifetime and max_cache_ttl. A paste that expires at time T must never remain readable from either cache after T.
Each cache layer can serve a live paste, but the expiry clock limits both cache lifetime and origin visibility.
Follow the read from the API-local L1 cache through Redis L2 and the expiry lookup to S3; the clock shows that the cache lifetime ends at the paste expiry, while concurrent misses merge into one origin fetch.
Layering hot-key protection
A viral paste can overload each API server even when Redis is healthy: every request still crosses the network to the shared cache. An optional in-process L1 cache absorbs repeated requests locally, with Redis acting as L2 for misses. L1 reduces shared-cache traffic, but each API server now has its own copy. Keep its expiry no later than the paste expiry, and invalidate or bypass it when deletion or another content change makes the cached value unsafe. The tradeoff is lower latency and less Redis traffic in exchange for more copies to expire or invalidate.
Consider a paste that goes viral five minutes before its one-hour expiry. If you insert it with a fixed one-hour cache TTL, requests can continue receiving the cached content after the paste should produce 410 Gone. With an expiry-aware TTL, the entry receives only the remaining five minutes. At the expiry boundary, both L1 and Redis reject the value, and the metadata check returns the expired result instead of reviving the paste.
Collapsing cold misses
Expiry-aware TTLs prevent stale hits, but they can make many requests miss together when a hot entry expires. If every request independently fetches the same object from S3, the cache has merely moved the bottleneck to the origin. Use request coalescing, also called a single-flight pattern: the first request becomes the loader for that key, while concurrent requests wait and retry the cache after the loader stores the result. This bounds one burst to one origin fetch per key rather than one fetch per request. Probabilistic early expiry is another option: selected requests refresh before the deadline, spreading replenishment instead of allowing a synchronized expiry.
CHECK YOUR UNDERSTANDING
A paste has a one-hour cache TTL but is configured to expire in ten minutes. What does a request at minute 11 see, and how do you fix the design?
SHOW ANSWERHIDE ANSWER
With the fixed cache TTL, the request can hit cached content even though the paste expired at minute 10, producing stale content instead of 410 Gone. On population, compute the remaining lifetime as expires_at - now and set the cache TTL to the smaller of that value and max_cache_ttl. Every L1 and Redis hit must also verify the stored expiry, so the entry is rejected at minute 10 even before eviction.
Anonymous deletion and short-ID safety
Anonymous pastes still need an owner-equivalent capability. At creation time, the service issues a deletion token and returns it alongside the short ID. The client must present that token to DELETE /pastes/{id}; the public ID alone is only a locator, not proof of ownership. Anyone who knows a paste URL can retrieve it in the MVP, but knowing that URL must not grant deletion rights.
Make deletion an explicit authorization check
The delete handler should resolve the public ID, validate the supplied deletion token against the token associated with that paste, and reject the request when they do not match. A successful deletion should make subsequent retrieval behave as deleted rather than silently recreating or replacing the paste. Keep the metadata uniqueness rule in force after deletion: if cleanup is asynchronous, a new paste must not reuse an identifier while an old record can still win a concurrent write.
Use a seven-character base-62 namespace for generated IDs. It contains approximately 3.5 trillion possible values; against the roughly 100 million stored pastes in this design, that is about 35,000 possible IDs per stored paste. The namespace is therefore ample, but randomness is not a uniqueness guarantee. Before publishing an ID, the service must make the identifier claim through the metadata store's uniqueness constraint. If the claim loses a race, generate another ID and retry. For a caller-supplied alias, do not retry with a different alias: return a collision response and leave the existing paste untouched.
This check must be atomic. Two create requests can both observe an alias as available if they perform a separate read followed by a write. Availability is only advisory; the uniqueness decision at insertion is authoritative. The winner persists the paste and receives its deletion token. The loser receives a collision response and must not receive a token for, or overwrite, the winner's paste.
Uniqueness is decided at the write, not by the preceding availability check.
Follow the two lanes to see both requests pass the availability check, only one win the atomic uniqueness decision, and only the winner receive a deletion token.
CHECK YOUR UNDERSTANDING
Two clients submit the same custom alias at nearly the same time. What prevents one paste from replacing the other?
SHOW ANSWERHIDE ANSWER
Both may pass an availability read, but only one can satisfy the metadata store's atomic uniqueness constraint. The other create fails with a collision response; it must not overwrite the existing row or receive a deletion token for it.
Bottlenecks and scaling order
The first pressure point is the read path. At the estimated workload, 10M new pastes per day is about 115 writes per second, while a 10:1 read-to-write ratio produces about 1,150 reads per second—10 times as much traffic. Scale API servers horizontally first when request latency or API errors rise; this is the least disruptive fix, but it adds instances to operate and makes cache coordination more complex.
After API capacity, investigate concentration rather than average QPS. A viral paste can make one key much hotter than the aggregate suggests. Keep Redis as the L2 cache, add a small in-process L1 cache on each API server, and coalesce concurrent cold misses so one request fetches the paste from the origin while the others wait for that result. The cost is more memory, invalidation complexity, and coordination between requests.
Use the first observable symptom to choose the least disruptive scaling change, then account for the new operational cost.
Bottleneck
Trigger
Observable symptom
Mitigation
New tradeoff
API capacity
Read-heavy traffic reaches API-server capacity
Rising request latency and errors at the API layer
Scale API servers horizontally
More instances to operate and more cache-coordination complexity
Redis hot keys
A popular paste concentrates requests on one Redis key
Redis is healthy, but API latency spikes for that paste
Add an in-process L1 cache in front of Redis L2; use request coalescing on cold misses
More cache memory, invalidation complexity, and single-flight coordination
Object-store reads/egress
Cache misses or a traffic spike send repeated reads to S3
Higher object-store request volume, transfer cost, and origin latency
Cache hot paste content in Redis; put a CDN in front of the read path for extremely popular pastes
More cache or CDN transfer cost and another layer to invalidate
Metadata lookup and expiry scans
Expiration checks and cleanup scans grow with the metadata set
Database load rises and expiry processing falls behind
Keep metadata separate from content; index or partition expiry data and run cleanup periodically
Additional database indexing or partitioning work and scheduler capacity
Cleanup backlog
Expired metadata and object deletion complete at different times
Expired S3 objects remain after metadata cleanup, or metadata points to deleted content
Mark expired metadata, enqueue deletion, and use an S3 lifecycle rule or asynchronous purge
More cleanup-job capacity, retries, and reconciliation logic
ID/alias collisions
Random short-key generation or a custom alias is already present
Create requests retry or reject because the key is taken
Use pre-generated keys from a key-generation service, or check for collisions before committing
A separate key store or collision-check I/O adds write-path coordination
The storage boundary is also a scaling decision. The estimated 100 GB of new content per day compounds to roughly 35 TB in a year—about 365 days of accumulation—so keep paste content in object storage and keep only short metadata in the relational database. This avoids making replication, backups, point-in-time restores, and metadata queries carry the blob volume. If cache misses or traffic spikes still drive object-store request and transfer pressure, extend the read path with a CDN; the tradeoff is additional transfer cost and another layer whose contents must be invalidated.
Later bottlenecks and deployment choices
When expiry checks and cleanup scans begin falling behind, index or partition expiry data and increase scheduler and cleanup-job capacity. Mark metadata expired, enqueue object deletion, and reconcile failures; the database and object store do not provide one atomic transaction, so the design can temporarily leave an orphaned object or metadata pointing at missing content. That failure is contained by retries and reconciliation, not eliminated.
Single-region deployment keeps consistency and operations simpler; global deployment improves geographic availability and latency but adds replication and consistency decisions. Choose deliberately whether an expired or deleted paste must disappear everywhere immediately, or whether higher availability permits a bounded convergence window. Each scaling step should follow an observed symptom: API saturation, a Redis hot key, origin request or egress growth, expiry-scan lag, then deletion backlog.
CHECK YOUR UNDERSTANDING
A breaking-news paste receives 50,000 requests in 60 seconds. Redis is healthy, but latency for that paste spikes. What is happening, and what should you do?
SHOW ANSWERHIDE ANSWER
The hot key is concentrating work at the API layer or on the path between the API and Redis; a healthy Redis node does not prove that the whole request path is healthy. Add an in-process L1 cache in front of Redis L2 so each API server can serve repeated requests locally, and coalesce concurrent cold misses so only one request fetches the paste from object storage. For an exceptionally popular paste, put a CDN in front of the read path, accepting added transfer cost and invalidation complexity.
KEY TAKEAWAYS
Size the first version around roughly 115 writes/second and 1,150 reads/second from 10 million daily pastes and a 10:1 read-to-write ratio.
Keep paste bytes in object storage and lifecycle metadata in a database so blob capacity and metadata queries scale independently.
Treat expiry as part of the cached value: every cache layer must reject content once the paste's expiry time passes.
Use a uniqueness constraint as the authoritative check for generated IDs and custom aliases; availability reads alone are race-prone.
Use layered caching and request coalescing to protect the origin from hot keys and synchronized cache misses.