Video summary

Fundamentals of Backend Architecture - How to Design Scalable Software

Main summary

Key takeaways

Educational

Main Ideas / Concepts Covered

  • Software architecture matters more than “how to build”

    • A good architecture makes software faster and cheaper to extend, and prevents business harm when production needs change.
    • The talk frames architecture as intentional decision-making (pros/cons), not premature complexity.

  • Start simple, then evolve through a realistic scaling path

    • The video proposes a gradual evolution of an architecture while learning the reasoning behind each step.
    • Use case: a Google Drive–like backend that supports uploading, downloading/serving files, starting small and scaling up.

  • Separation of concerns

    • Server serves requests; database stores data.
    • Key lesson early: data coupled to a machine ⇒ no scale and no resilience.

Methodology / Step-by-Step Progression

1) Single server + in-memory/server-local data (baseline)

  • Architecture

    • A single server instance handles requests.
    • All file data is stored inside the server (e.g., a map of file ID → location).

  • Problems

    • If the server dies, data is lost.
    • Scaling by adding servers causes duplicated data and synchronization problems.

2) Add a database to decouple data from servers (persistence + stateless servers)

  • Change

    • Make the server stateless and persist file data in a relational database.

  • Result

    • Data persists across server restarts.
    • Multiple servers can scale without duplicating the source of truth.

  • Why

    • Achieves separation: servers focus on serving, DB focuses on storing.

3) Horizontal scaling with a load balancer (traffic distribution)

  • Change

    • Run multiple server instances.
    • Introduce a load balancer to route incoming requests to healthy servers.

  • Load balancing approaches discussed

    • Round-robin: alternate servers sequentially.
    • Health checks: route more traffic to healthier/less loaded servers.
    • Smart routing: route by API path or purpose (e.g., send upload-heavy traffic to a specialized file-upload service).

  • Session affinity / sticky sessions

    • Purpose: keep the same user on the same server when servers maintain per-user state (e.g., session data).
    • Demonstrated idea: load balancer uses a session cookie/token to maintain affinity.

4) Vertical vs horizontal scaling + autoscaling

  • Horizontal scaling

    • Add more machines to handle more requests.

  • Vertical scaling

    • Use a “beefier” single machine with more CPU/RAM/storage.

  • Autoscaling

    • Load balancer/cloud can adjust the number of instances based on traffic demand.
    • The video mentions managed/serverless approaches where provisioning is handled by the provider.

5) Move to microservices for specialization (as the system and team grow)

  • Motivation

    • As usage grows and the org scales (more engineers/teams), some endpoints become bottlenecks.
    • Microservices can isolate and optimize services with different workloads.

  • Example services for the Google Drive clone

    • File service: handle uploads (and likely file-related operations)
    • Notification service: push/web/desktop notifications
    • Auth service: authentication (login/token issuance)
    • Real-time service: syncing across devices (cloud sync)

  • Load balancer + routing limitation

    • Not all routing decisions are simple; authentication and cross-service flows need deeper coordination.

6) Add an API Gateway as the central entry point

  • Problem addressed

    • Prevent clients from calling services directly.
    • Centralize routing, aggregation, and security handling.

  • Gateway responsibilities

    • Receive requests
    • Analyze/route to the correct downstream service(s)
    • Aggregate responses (e.g., auth + profile info)
    • Act as the only public entry point

  • Network/security model

    • Services live in a private network (e.g., VPC concept).
    • Only the gateway exposes externally; services do not.

  • Gateway scaling note

    • Gateway could become a bottleneck / single point of failure; it may need scaling and possibly an upstream load balancer (mentioned conceptually).

7) Authentication and authorization in distributed architecture

  • Token-based authentication at the gateway edge

    • Clients send a JWT-like token (authorization header/cookie concept).
    • Gateway verifies token validity and can return 401 when invalid/expired.

  • Auth service role

    • Issues tokens after validating user credentials.
    • Uses a private key to sign tokens and may validate user existence (e.g., via a user-related service).

  • Authentication vs authorization

    • Authentication: “Are you who you claim?”
    • Authorization: “Do you have permission to perform the action?”
    • Authorization can be implemented per service (each service enforces its own permission rules).

8) File upload pattern using object storage (avoid pushing large blobs through servers/DB)

  • Why not upload directly to relational DB

    • Relational DBs are not designed for large blobs (e.g., 200MB–20GB).

  • Why not stream file bytes through application servers

    • Performance and security considerations (reduces attack surface and resource exhaustion).

  • Recommended pattern

    • Use object storage (e.g., S3/GCS bucket) for the actual file bytes.
    • Flow:
      • Client calls gateway/API with metadata (file name, size, type, etc.).
      • Metadata/service + database creates a new file record and ID.
      • System requests object storage to generate an upload URL.
      • Upload URL has an expiry window (small time validity; may also restrict size).
      • Client uploads bytes directly to the bucket (servers bypass for the large payload).

  • Result

    • Upload traffic no longer “bombards” application services.
    • Services mainly handle metadata and orchestration.

9) Event-driven processing with a broker/message system (reliability + fan-out)

  • Problem with synchronous calls

    • After upload, services may need follow-up actions (thumbnail generation, real-time sync, notifications).
    • Direct synchronous calls can fail if a service is down/slow, leaving missing derived outputs (e.g., videos without thumbnails).

  • Solution

    • Use a broker/queue/event bus (Kafka/RabbitMQ-like concept).

  • What changes

    • Object storage emits an event after upload.
    • Broker delivers events to subscribing services (thumbnail service, real-time, notification, etc.).

  • Broker reliability features

    • High availability
    • Durable message storage
    • Acknowledgements and redelivery
    • Dead letter queue (DLQ) for messages that can’t be delivered
    • Alerts/monitoring (e.g., Slack/Discord) for failed messages

  • Core takeaway

    • At larger scale, separation of concerns improves when services react to events rather than coordinating via fragile direct calls.

10) Scaling optimizations: caching, CDN, and rate limiting (as traffic grows)

  • Caution

    • Don’t over-optimize prematurely; this comes after scaling is working.

  • Caching targets

    • Metadata caching (e.g., file name/properties), since it rarely changes.
    • Avoid storing large blobs in in-memory caches (noted limitation).

  • Cache pattern described

    • Lookup key in cache:
      • If present: return cached result
      • If absent: fetch from DB/object storage (or DB for metadata), then store in cache, then return

  • CDN for large assets

    • CDN sits at the edge near users globally.
    • First request may hit origin; subsequent requests are served from edge cache, reducing latency significantly.

  • Rate limiting

    • Purpose: prevent malicious or abusive traffic from exhausting resources and increasing costs/impact.
    • Mechanism:
      • Track requests per client identity (IP/user/token).
      • Use fast storage (often cache/Redis-like) to count requests in a time window.
      • Exceeds threshold ⇒ return HTTP 429.
    • Analogy/implementation detail:
      • Similar to “token” limits (like quota/currency), though described more simply with request counts.

Speakers / Sources Featured

  • Speaker

    • Not explicitly named in the subtitles (the narrator/creator is referred to as “I” throughout, but no name is given).

  • Referenced source(s)

    • Martin Fowler (referenced via his work and an article/phrase; exact article name not provided).

  • Referenced books

    • One general “favorite” book is mentioned, but not titled in the subtitles.

Original video