The Staff Engineer Caching Interview: Invalidation, Stampedes, and the Origin You Must Protect

“Put Redis in front of the database” sounds like the end of a caching discussion. In a staff engineer interview, it is the beginning. The harder question is what happens when a popular key expires during a traffic spike, a price changes while a page is being served, or a personalized response lands in a shared cache. Amazon’s caching guidance describes all three classes of risk: downstream overload, growing inconsistency, and security exposure.

A useful answer treats the cache as a second serving system with its own state, failure modes, and operating cost. You must say which data can be stale, which system owns the truth, how misses reach that system, and how you will know the design still works when traffic changes. This guide builds that answer around an illustrative product-detail API. The example’s workload and timing choices are interview assumptions, not measured production data or claims about any employer’s interview rubric.

Begin with the freshness contract

Before you draw boxes, ask what the cached response means to the user. A product page can often tolerate a brief delay before a revised description appears. A checkout decision about inventory cannot safely depend on a browser’s old view of stock. The checkout service must consult an authoritative store and enforce its purchase invariant there. That is a design choice based on consequence, not a claim that every product catalog has the same requirements.

Separate display freshness from transaction correctness

State the contract aloud: “Catalog text may be stale for a bounded interval; availability on the page is advisory; a purchase is confirmed only by the authoritative transaction.” If the interviewer changes the domain to account permissions or a trading balance, tighten the contract. HTTP caching rules distinguish responses that can be reused while fresh from responses that must be validated and prohibit a cache from ignoring must-revalidate. Amazon likewise says cache suitability depends on whether clients tolerate eventual consistency and on how fast the source changes. Those requirements should determine the TTL, not the other way around.

Ask for four facts: the acceptable staleness of each field, the read-to-write shape, the cost of a miss, and the consequence of a stale or wrong response. Do not collapse “fresh enough for browse” and “correct enough to commit money” into one cache policy. A sharp answer can allow stale catalog details during an origin incident while refusing to complete a purchase without an authoritative stock check. Google SRE describes serving a cheaper local copy as a degraded response when that tradeoff fits the service.

Choose where the cache lives

“The cache” may mean a browser, a CDN edge, an application process, or a shared in-memory service. They solve different problems. Browser and CDN caches save network trips and origin work for cacheable HTTP responses. An in-process cache removes a network hop inside one service instance. A shared cache lets instances reuse a common working set. Amazon’s comparison of local and external caches notes that local caches warm separately on each instance and can give successive requests different versions of the same value.

For our product page, send immutable images and versioned assets through the CDN. Consider edge caching for anonymous catalog responses whose freshness contract permits it. Keep personalized recommendations or account-specific pricing out of a shared response cache unless the key and access rules explicitly account for the viewer. At the application layer, cache repeated reads of product summaries if their miss cost justifies another dependency. This is a placement proposal to test with real traffic, not an automatic requirement to add every layer.

If the interviewer asks why an edge cache helps the origin, explain the hierarchy. Cloudflare’s tiered cache makes lower-tier locations ask an upper-tier cache before the origin; only an upper tier fetches from the origin on a miss. But extra layers complicate freshness and invalidation. Draw every hop and label which one can serve a stale value and which one can bypass caching.

Choose a read and write pattern

For a read-heavy product catalog, cache-aside is a defensible starting point: read the cache; on a miss, load from the primary source; populate the cache with an expiration; return the result. On a successful source write, remove the relevant cached entry so the next read reloads it. Redis documents this flow and contrasts it with write-through and write-behind approaches. The attraction is that only requested data fills the cache. The cost is that misses and races still need explicit handling.

Do not turn a cache into the source of truth by accident

Consider a write-through design if every change should update cache and database together through one abstraction. Say how partial failure is resolved: if the database commits and the cache update fails, which value will readers see? If you cannot make both stores atomic, invalidation after the durable write plus a bounded TTL is simpler to reason about. A write-behind queue can make writes appear fast, but it creates a period in which the authoritative store has not recorded the change. That is a poor default for stock or payment state. The right comparison is the failure semantics of each pattern, not the number of arrows in the diagram.

Make the miss path concrete. Read from the source with a deadline, record the miss separately from a source failure, and decide whether a known last-good value is acceptable. Amazon describes soft and hard TTLs: attempt refresh at the soft boundary, then serve an older value only up to a hard boundary when the downstream source is unavailable. Apply that pattern only to data for which stale serving is allowed by the contract.

Design keys before tuning TTLs

A cache key must include every input that changes the response. Product ID may be insufficient when currency, locale, region, catalog version, or customer segment changes the body. Leaving one out risks returning the wrong representation; adding irrelevant, high-cardinality fields wastes memory and lowers the hit rate. CloudFront’s cache-key guidance makes that tradeoff explicit, and Cloudflare documents how query strings, headers, cookies, and user features alter keys.

Use a schema you can explain, such as product:v3:{region}:{currency}:{id}. Versioning provides a controlled way to stop reading an old serialized shape after a deployment. Normalize inputs before making the key: otherwise semantically identical requests may fragment the cache. Redis uses application-level naming conventions because its keys are just strings; its keyspace guidance describes colon-separated names and per-key expiry.

At the HTTP layer, do not confuse “stored” with “fresh.” RFC 9111 defines Vary, freshness, and revalidation. RFC 9110 defines conditional requests: an If-None-Match check can produce a 304 Not Modified when the representation has not changed. Revalidation can save a body transfer, but it still requires work by the server. If a CDN’s minimum TTL overrides the origin’s intent, the edge may retain a response longer than expected; CloudFront explicitly warns that a nonzero minimum TTL can override no-cache, no-store, and private.

Keep a miss from becoming an outage

A cache hit ratio is an average; the origin experiences the misses. If a popular key expires while many requests arrive, all of them can query the database at once. The same burst happens when a new fleet starts with empty local caches or a cache cluster is flushed. Amazon calls this a thundering herd and recommends request coalescing so one pending load serves concurrent misses for the same resource. CloudFront also describes request collapsing for simultaneous requests sharing a cache key.

Make the origin load predictable

In an application cache, use singleflight or a short-lived per-key lease around the loader. Redis SET with NX and an expiry provides a primitive for acquiring a lease, but a production answer must also cover lease expiry, owner identity, and what waiters do if the loader fails. Do not claim a simple lock provides perfect consistency. Add bounded source concurrency and a timeout so a miss storm cannot consume every database connection.

When the freshness contract permits, serve a recent value while one worker refreshes it. RFC 5861 defines stale-while-revalidate and stale-if-error; Cloudflare documents how it serves stale content during background revalidation when the origin sends the directive. Spread refresh times or prewarm a small set of genuinely hot keys if measurements justify the extra work. Then test a full cache failure, because a design that works only while the cache is healthy has made its origin more fragile.

Make invalidation part of the write path

“We’ll expire it after five minutes” may bound staleness, but it does not answer what happens just after a price change. Name the source write, the cache action, and the race between them. For cache-aside, a simple sequence is to commit the authoritative write and then delete affected keys. If deletion fails, an old value may remain until TTL; if a concurrent miss loaded an old value before the commit and stores it afterward, a stale value can reappear. Explain the residual window and whether the contract accepts it. Redis’s cache-aside example uses explicit invalidation on write plus expiration.

If the contract requires tighter bounds, consider versioned keys tied to a database revision, an ordered invalidation event, or reading the source for the critical action. Each adds complexity. Say which failures you will replay and how you will verify propagation. For assets, versioned names can be cleaner than broad purges: CloudFront recommends versioning for frequently updated files because browser and intermediary caches can keep the old URL even after an edge invalidation.

For dynamic content, a targeted purge may be appropriate. Cloudflare offers URL, tag, prefix, host, and full-cache purges, and notes that a successful purge API response confirms receipt, not that a particular object was previously cached or removed. Verify the next response and its cache status. A staff-level answer explains who owns that verification and what happens if the purge call succeeds but users still see an old value.

Protect tenants and personalized responses

A shared cache is an access-control boundary. If a response varies by account or tenant, the cache key must represent that boundary, or the response must bypass the shared cache. Never assume that a CDN will infer personalization from an application session. CloudFront says response-changing headers and cookies need deliberate cache-key treatment; RFC 9111 distinguishes private responses from those reusable by shared caches.

Check the actual CDN settings, because platform rules can alter defaults. Cloudflare’s default behavior excludes responses with private, no-store, or Set-Cookie, subject to configuration such as an edge TTL override. Amazon’s caching guidance adds risks of sensitive data exposure, cache poisoning, and timing side channels. Those risks belong in the interview answer, especially for account data. OWASP explains how a poisoned response in a shared cache can spread malicious content to many users.

In the product-page scenario, anonymous product descriptions may be public; negotiated customer prices may not be. Keep the latter private or require a tenant-aware key and authorization at the source. Test with two users whose responses differ, then repeat after a cache hit. If one user sees the other’s value, the system has failed regardless of its latency.

Prove the cache helps under load

Measure the cache and the dependency together. Track hit and miss rates by endpoint and key family, evictions, memory pressure, loader concurrency, origin QPS, origin latency, and the age of served data where freshness matters. Amazon recommends hits, misses, cache size, and downstream requests; Redis’s eviction guidance points to evicted_keys and expired_keys when tuning memory and TTL. A high hit rate can conceal a disastrous concentration of expensive misses, so look at miss cost, not just the percentage.

Size from a measured working set and key distribution, then leave headroom for metadata, replication, and bursts. An eviction policy is a workload choice: Redis documents LRU, LFU, and TTL-sensitive options under maxmemory. Its memory guidance also warns that resident memory may stay high after keys are removed. Avoid sizing a production fleet solely from serialized payload bytes.

Run the unpleasant tests: cold startup, a hot-key expiry, a regional edge miss, a failed invalidation, and a cache outage. Observe whether the origin can survive the resulting load. Amazon specifically advises load tests with caches disabled. Google SRE notes that cache misses may be much more expensive than hits and can contribute to cascading failure. If the origin cannot carry all misses, state the backpressure, stale-serving, and load-shedding plan before claiming the design is resilient.

Walk through a staff-level answer

Suppose an interviewer asks: “Design a product-detail service for heavy read traffic. Editors update descriptions, prices change, and customers see inventory. What would you cache?” Work through the answer in this order. The volumes and TTLs below are choices to negotiate with the interviewer; they are illustrative assumptions, not published benchmarks.

1. Define correctness. Product descriptions can be a few minutes old if the business accepts it. A displayed inventory count is advisory. Purchase confirmation uses an authoritative transaction. Ask whether personalized prices exist before defining a shared key.
2. Separate layers. Serve versioned images and static assets at the edge. Cache anonymous product summaries at the edge only if headers and key dimensions express their variants. Use an application cache for repeated database reads if the origin workload warrants it.
3. Show the read path. On a hit, serve the eligible value. On a miss, coalesce concurrent loads for the same key, read the authoritative source within a deadline, and populate the cache with a bounded expiry.
4. Show the write path. Commit the source change, invalidate the affected summary key, and record any failure to invalidate. Keep the checkout invariant in the source transaction.
5. Show failure behavior. If the cache is down, bound fallback concurrency. If the origin is down, serve only fields whose stale-data contract permits it; otherwise return a clear error. Use a hard expiry rather than serving old information indefinitely.
6. Show proof. Compare origin load and user latency with and without the cache. Test the hot-key expiry and two-user isolation. Inspect real cache-status headers at the CDN and miss/eviction metrics in the application cache.

Do not add a numerical TTL merely to sound decisive. First ask how quickly each field changes and how much inconsistency the product will tolerate. Then propose a starting TTL, a way to measure stale exposure, and a revision rule. CloudFront’s expiration rules show why both origin headers and cache-policy bounds matter. Cloudflare’s troubleshooting guide shows how HIT, MISS, and BYPASS help check what happened, rather than what a diagram predicted.

Practice the follow-up questions

The interviewer’s most useful interruptions change one constraint. “The largest merchant gets a cache miss storm.” “The price is customer-specific.” “The cache cluster is unavailable.” “Editors need changes visible immediately.” “Your hit rate is high, but the database is still overloaded.” Answer each by returning to the original contract, naming the part of the system that changes, and specifying the evidence you would collect.

For the miss storm, coalesce loads and cap origin concurrency. For customer-specific prices, bypass a shared edge response or key it by a safe authorization boundary and test isolation. For immediate editorial visibility, use targeted invalidation or versioned keys and verify propagation. For a healthy hit rate with a struggling database, inspect which misses are expensive and whether a few hot keys or cold starts dominate. A cache that reduces average latency while making the worst day harder to survive is not a complete design.

The distinctive staff-level move is to make the cache accountable to the service’s promises. When you can explain exactly what users may see, what the origin must withstand, and how you will detect a broken assumption, the technology choice becomes a consequence of the design. That is the answer worth rehearsing.