Video summary

What are Distributed CACHES and how do they manage DATA CONSISTENCY?

Main summary

Key takeaways

Technology

What the video explains about caching (and distributed caches)

Goal of caching

Caching is used to improve response time for clients by serving data from a cache instead of repeatedly querying a database.

Two main reasons to use a cache

  1. Reuse commonly requested data Cache stores key → value mappings in fast storage (often memory) so the system avoids repeated network/database calls (e.g., user profiles).

  2. Avoid expensive repeated computation/aggregation Cache results of costly operations so they’re computed once and reused (e.g., compute an average age once instead of recomputing from all users every request).

Why you shouldn’t “cache everything”

  • Cost: Cache hardware is often more expensive than database hardware (commonly SSD-based).
  • Performance tradeoff: If the cache grows too large, lookup/search time increases, reducing the benefit.
  • Practical requirement: The cache should store the most relevant data likely to be requested soon, based on eviction and loading decisions.

Cache policy (how entries are chosen)

The video emphasizes that cache performance depends heavily on the cache policy, including:

  • When to insert/load entries into the cache
  • When to evict/remove entries

Policies mentioned

  • LRU (Least Recently Used): keep recently accessed items (“top”) and evict least recently used (“bottom”).
  • LFU (Least Frequently Used): evict based on access frequency (noted as less common/mentioned as an alternative).
  • Sliding window based policies: newer approaches; the video claims they can outperform LRU.

LRU intuition: “hot” items (e.g., trending comments) stay longer, while “cooling down” items eventually get evicted.

When caching backfires

  • Bad eviction policy: If the cache frequently misses, it becomes harmful—you still pay the cache lookup cost and end up hitting the database anyway.
  • Thrashing (small cache issue): If the cache can’t hold the working set of data, requests continually evict each other’s entries, so you never benefit from cached results.

The core distributed-cache issue: data consistency

The key problem in distributed caching is consistency:

  • One server updates the database (e.g., user profile X), but other servers’ caches may still contain stale values.
  • Those servers can serve outdated information (e.g., an old password still working), creating correctness/security risks.

Example risk: caches on other servers may not reflect the latest database update.

Where to place the cache (local vs global)

Local cache (close to servers / in-memory on each server)

Pros

  • Very fast
  • Simpler design

Cons

  • Cache state is tied to server memory—a server crash wipes its cache.
  • Consistency issues can arise when caches diverge across servers.

The video notes that correctness tolerance varies by data type (e.g., profiles vs critical data like passwords/financial info).

Global distributed cache (separate system, e.g., Redis)

Pros

  • Resilience: if an application server crashes, cached data may remain in the global cache.
  • Independent scaling: the cache can scale separately from app server memory.
  • Centralized handling can reduce “consistency headache.”

Cons

  • Typically slightly slower than per-server in-memory caches.

Redis is referenced as a common distributed cache example, used to store key/value data for fast retrieval.

Consistency strategies: write-through vs write-back

Write-through cache (cache + update database immediately)

  • On update, the system writes/propagates the change so cache and DB remain aligned.
  • In multi-cache/server setups, other servers may still have stale cached copies unless invalidated/updated.

Write-back cache (update DB via cache asynchronously)

  • The database is treated as the source of truth, while cache synchronization may lag.
  • The video warns that write-back can be expensive: keeping consistency may require invalidating many entries, causing frequent database reads—described as behavior that can resemble thrashing.

Hybrid approach (best of both worlds)

The video suggests a hybrid mechanism:

  • For non-critical updates: update cache first and persist to the database later in batches to reduce database calls.
  • For critical data (e.g., passwords/financial): use stricter approaches (closer to write-through/write-back requirements, as implied).

Main speakers/sources

  • Main speaker: the video narrator/host (no specific name provided in subtitles).
  • Referenced source: “a Google developer who made caffeine” (sliding window policies claim) — specific name not provided in subtitles.

Original video