Cloudflare saved 100 terabytes with just 5 Rust optimizations

Overview

Cloudflare’s Rust DNS cache (“Big Pineapple”) was redesigned to reduce memory usage and improve performance using five Rust-focused optimizations. The techniques are broadly applicable to writing performant Rust.

• The cache starts empty, fills as DNS queries arrive, and evicts older/less-used entries.
• Each cache entry stores:
  • Query-related keys (e.g., query name/type/metadata)
  • DNS response data (e.g., timestamp, TTL, authority/additional records, errors, etc.)
• The post highlights that many struct fields/types carried unnecessary overhead.

---

Key optimizations (Rust / design changes)

1. Stop using Vec for fixed-size data

Problem: Vec stores a pointer plus length and capacity, and unused capacity wastes memory (e.g., capacity 8 but only 5 items used).

Change: Replace vectors with Boxed heap-allocated fixed-size arrays, removing wasted capacity fields and reserved slots.

---

2. Reduce number/shape of collections in the cache entry

Change: Replace separate answers, authority, and additional lists with a single records list plus offsets marking where authority/additional segments begin.

Result:

• Fewer heap allocations: 3 → 1
• Smaller metadata: 48 bytes → 20 bytes

Also included:

• Pack multiple boolean fields into a single bit flag.

---

3. Optimize for the most common case by dropping most owner data

DNS records include an owner (the domain the record belongs to), which is frequently the same as the queried domain.

Change: Store owner as an Option:

• Usually None (owner can be derived)
• Store an owner string (with heap allocation) only in minority cases

---

4. Box large enum variants to avoid padding

In Rust, enums use the size of their largest variant, so smaller variants can waste space due to padding (described as >100 bytes padding).

Change: Box the largest enum variants (the “last three variants” mentioned in the described enum) to reduce padding dramatically.

• Worst-case reduced to ~12 bytes

Trade-off: boxing adds allocator overhead and can reduce locality.

---

5. Store record data in a contiguous “wire format” buffer

Instead of storing records as struct allocations, store them in a contiguous byte buffer with a 2-byte length prefix per record.

Benefits:

• Removes previous heap-allocation patterns
• Improves CPU cache locality

Trade-off: records can’t be randomly indexed; lookups require scanning the buffer.

Overall theme: restructure data to minimize allocations and wasted space while improving cache locality.

---

Rollout timeline and measured results

• All five optimizations were deployed over ~2 months.

After rollout:

• Per-entry footprint: -56%
• Per-entry allocations: -58%
• Cache insert throughput: +43%
• Cache lookup latency: -19%
• Overall memory usage: ~100 TB less

---

Tutorial / review style elements

• This is presented as a tech breakdown of a real-world Rust optimization case study (Cloudflare blog post), not a step-by-step tutorial.
• A “learn Rust professionally” section promotes a mentorship program, but the technical content focuses on the five cache/data-structure optimizations above.

---

Main speakers / sources

• Source: Cloudflare engineering blog post (referenced as the “Big Pineapple” DNS cache / optimization case study).
• Speaker / CTA voiceover: host promoting “Rust Live Accelerator” and directing viewers to:
  • letsgetrusty.com/join
  • (plus a pinned-comment link)