Caching Is a Consistency Problem

Caching gets introduced into systems as a performance optimization, discussed in terms of hit rates and latency percentiles. That framing hides the actual engineering problem. The moment you cache a value, you have created a second copy of some fact, and now your system has two places that might disagree about what’s true. Caching is not a performance problem with a consistency side effect — it’s a consistency problem with a performance benefit.

The copy you didn’t think you made

Consider the most ordinary caching decision: storing a user’s profile in memory after reading it from the database, so the next request doesn’t hit the database again.

Source of truth:  database row for user 42
Cache:            in-memory copy of user 42, read at time T

The instant that copy exists, a question exists alongside it that didn’t exist before: what happens when the database row changes at time T + 1? Every answer to that question is a real trade-off, not a detail to defer:

• The cache doesn’t know, and serves the stale copy until it expires. You’ve traded correctness for simplicity, bounded by your TTL.
• The write path explicitly invalidates or updates the cache. You’ve traded simplicity for correctness, and now the write path has to know about every cache that might hold a copy of what it’s writing.
• The cache is never trusted for anything that must be current, and is only used for data where staleness has a defined, acceptable cost. This is the honest version of option one — it doesn’t remove the trade-off, it scopes it deliberately.

There is no fourth option where caching is free. Every cache is a bet about how much staleness a specific piece of data can tolerate, made explicit or made by accident.

Where this goes wrong in practice

Most caching bugs aren’t algorithmic — they’re a case where the trade-off above was made implicitly instead of deliberately. A few recurring shapes:

The cache outlives its invalidation trigger. A value is cached with a long TTL for performance, and the write path is updated later to change how that value gets computed — but nobody re-examines whether the old TTL is still a safe bet against the new write frequency. The cache becomes wrong more often, silently, because the trade-off was set once and never revisited against a changing system.

Multiple caches, one invalidation path. A value gets cached at more than one layer — in application memory, in a shared cache, in a CDN — and the invalidation logic only knows about one of them. The system now has three copies of the truth and one invalidate() call, which is a consistency bug waiting for the right timing.

Cache used as a substitute for a slow query, in a place that needed correctness. This is the deterministic-boundary problem from a different angle: a fast, sometimes-wrong answer got substituted for a slow, always-right one, in a place where the difference actually mattered — a rate limit check, an authorization decision, an inventory count. The performance win is real. So is the correctness we gave up to get it, and it’s rarely written down anywhere that we gave it up on purpose.

The question every cache should be able to answer

A cache that was designed deliberately, rather than added reflexively, can answer three questions:

1. What is the maximum acceptable staleness for this data, and why is that the
   right number? (Not "what TTL did we pick" — what's the actual tolerance.)
2. What invalidates this cache, and does every writer to the source of truth
   know about that invalidation path?
3. What happens to a reader if the cache and the source of truth disagree —
   is stale-and-fast always preferable, or are there callers that need
   correct-and-slow instead?

If a cache can’t answer all three, it isn’t a caching decision — it’s a caching guess, and it will eventually produce a bug that looks like it came from somewhere else entirely, because the actual defect (an unexamined staleness assumption) is invisible until the timing lines up wrong.

The generalization

This is really a special case of a more general problem: the cost of multiple sources of truth, wherever they show up — cached values, denormalized fields, derived data stored instead of computed, replicas that lag their primary. Every one of these is the same bet caching makes, just with a different name. Naming it as a consistency trade-off, instead of a performance feature, is what makes it possible to reason about deliberately instead of discovering the trade-off in production.

• #caching
• #distributed-systems
• #consistency