How to Build Your Own Data Center & Why Every Startup Should Do It

Business / strategy takeaways (Speechify + data center build-out)

• Speechify treats compute as a strategic asset (“ring at home”), not a variable rental cost—so engineers avoid bottlenecks and can iterate faster.
• Own GPUs for training and latency-critical workloads; rent/spot for variable inference and experimental work to balance cost and flexibility.
• Data center capability is an execution advantage, not just infrastructure:
  • Secure/own hardware placement near compute memory needs (large-scale training benefits from “memory next to GPU clusters”).
  • Use liquid cooling sidecars to reduce cold-start costs when retrofitting data centers.
  • Outsource logistics/maintenance to data center facilities (energy, physical engineers, network).

---

Frameworks / playbooks mentioned or implied

Cost math / ROI model (buy vs. rent)

• Renting H100-class GPUs via hyperscaler spot/discounted hourly rates is ~1.5× the yearly cost of owning (based on their illustrative numbers).

Capacity planning model (seasonality + commitments + flexibility)

• Treat monthly demand as a percentage of “100% capacity” (training + inference).
• Use a layered GPU allocation:
  • Buy for baseline/core usage
  • Secure some capacity via long-term hyperscaler contracts
  • Rent remaining needs via spot instances

Experiment throughput model

• Training speed matters because “every minute of delay” can affect competitive outcomes.
• Inference can use older/fewer-power GPUs since latency tolerances are typically looser (e.g., ~100 milliseconds acceptable).

Organizational incentive system (credit only after user impact)

• Avoid token/effort leaderboards.
• Reward what ships to users and works in production.

AI engineering team operating model

• Engineers act as “quality control specialists” for AI-generated outputs.
• Success depends on fast loops: goal → measure → iterate/check (loop-based execution).

---

Key examples / concrete cases

• Speechify 3.2 model positioning

  • Claimed: Speechify 3.2 “ranks first in the world in quality.”
  • Claimed pricing edge: “10× more affordable” than services like ElevenLabs/11Labs (as stated in subtitles).

• Competitive benchmarking / product iteration

  • Frequent releases (“we don’t do theory—we’re an applied AI company”).
  • Leadership personally tests features on a phone during Zoom calls to surface defects before credit is assigned.

• GPU procurement negotiation example

  • Paid extra ~$100k to accelerate GPU delivery by a month (supplier example in France).
  • Emphasizes that the most expensive part of shipping/delay is continuing data-center space cost.

• Cooling implementation

  • Since many data centers lack liquid cooling, they used liquid cooling sidecars installed on-site to enable liquid-cooled racks.

---

Key metrics and KPIs mentioned (and what they imply)

GPU / hardware economics

• Illustrative H100 purchase vs. rental

  • Buy: ~$30,000 per H100
  • Rent:
    • GCP Spot: ~$5/hour
    • AWS/Azure-style: ~$3.50/hour
  • Annual rental estimate: ~$35,000–$50,000 per year
  • Conclusion: renting ≈ 1.5× owning annually

• Warranty / useful life

  • Warranty: ~3 years
  • Expected continued operation: ~10 years (as stated)

Inference / inference cost benchmarks (token economics)

• Speechify constraint (B2C → B2B pivot)

  • Target/constraint: < $10 per million characters
  • ElevenLabs: ~$100 per million characters
  • OpenAI model tests (as stated): ~$196 per million characters

• Example monetization claim

  • Speechify API launch “Simba 3.2”: ~$10 per million characters cost-to-serve when selling to other B2B companies (per their statement)

Scale / usage

• User scale & inference volume

  • “60 million users” using Speechify inference features (text-to-speech).
  • App Store claim: 98% of B2C installs in the “text to speech” category.

• Content consumption scale

  • 770 billion words spoken by Speechify users (framed as ~6,000 years of listening time).

Team / resourcing

• Team size targets and structure:

  • Current AI engineering: 45 engineers
  • Goal: 150 engineers
  • Org constraint: “25% of my team is actually just waiting” → needs more “practice space” (capacity).

• Process throughput claim:

  • Engineers expected to produce “10 really good solutions a day” (as described).

---

Actionable recommendations (from their operational philosophy)

• Build your “ring at home”

  • Own compute for training and time-sensitive experiments to minimize iteration delay.

• Use a layered utilization strategy

  • Buy for baseline (training + always-on inference).
  • Contract for partial demand smoothing.
  • Rent/spot for variable remainder (seasonality/load spikes).

• Separate inference from training requirements

  • Use older GPUs for inference if latency targets are met.
  • Reserve newest architectures for learning/training.

• Over-communicate with customers as internal QA

  • Leadership should directly validate quality in the product (e.g., testing on phone) before giving technical teams credit.

• Avoid “token spend” waste via architecture + loop discipline

  • Optimize prompting/agent behavior to reduce unnecessary token burn.
  • Use loops: set goal + measure + repeat until quality improves.

• Hiring changes for the AI era

  • Use “functional interviews”: build first, then test with unit tests.
  • Give candidates a real codebase and see what they break.
  • Require agent-management capability for top roles.
  • Focus hiring on aptitude/intelligence and potential; “teach the rest.”

---

B2B go-to-market pivot (leadership + strategic correction)

Speechify’s “biggest strategic mistake”

• Going B2B later than competitors (notably compared to ElevenLabs/Sierra positioning).
• Reason then: thought TTS APIs would become commodity and B2B pursuit wouldn’t be worth early investment.
• Updated insight: an “AI lab” wedge (API product) enables continuous innovation and expansion into:
  • Multi-voice, voice cloning
  • Speech-to-text
  • Emotional intonation
  • Duplex/conversational capabilities
  • Agents

Why B2B matters operationally

• Original issue: B2C had too-low willingness to pay; they needed a cost-to-serve advantage.
• Now competition is economics-driven:
  • ~$10 per million characters cost-to-serve vs higher competitor costs (as stated)

Market positioning stance

• They argue customer-support-heavy B2B markets are crowded, making differentiation hard.
• They position their B2B strength as API + quality/speed/cost, and add agents only where needed.

---

Company infrastructure / “data center playbook” (end-to-end execution)

• Hardware

  • GPU racks delivered to their data center (e.g., “Rubens come as 72 cards in one rack”).
  • They also buy newer variants early (e.g., B300) when acquisition timing beats hyperscaler availability.

• Facility

  • Rent data center space for:
    • Network capabilities
    • Energy (biggest constraint)
    • On-site physical engineers for unloading/installation and troubleshooting

• Cooling

  • Prefer liquid cooling (Rubens are liquid-cooled).
  • If the facility isn’t equipped, use liquid cooling sidecars and staff installation.

---

High-level investing / markets notes (kept minimal)

• Nvidia/secondary market enabling finance was discussed:
  • Reported partnerships aimed at underwriting/guaranteeing GPU resale/buyback (secondary liquidity).
  • Framed as supporting broader GPU adoption rather than speculative hype.
  • “Intrinsic value” argument tied to compute throughput (teraflops per second).

---

Presenters / sources mentioned

• Cliff Weitzman — Founder & CEO, Speechify (primary speaker)
• Piotr — Person at ElevenLabs (mentioned as having built an initial home GPU cluster)
• Mati — ElevenLabs co-founder (mentioned as met in 2022)
• Brett Taylor — Co-founder of Sierra; discussed extensively
• Sierra — Company mentioned (customer-support/agent platform)
• Elon Musk — referenced in relation to GPU/space context (high level)
• Jeff Dean — referenced (excites them)
• Jason Yeager — referenced as running an Instagram account (“my tech CEO”)