System Design for Beginners Course

Summary: System Design Course (Technological Concepts, Product Features, Guides/Tutorial Flow)

Course overview & learning goals

• A beginner-focused system design course developed by Gaurav Sen.
• Part 1 teaches how to recognize engineering design patterns used to build large-scale distributed systems.
• Part 2 applies those principles by designing and coding a live streaming video application.

Key definitions: “large-scale distributed systems” and design patterns

• Large-scale: Systems with heavy compute/data, high usage, frequent updates, and strict performance expectations (e.g., Google Maps).
• Distributed systems: Execution/data are spread across multiple locations (regions/servers) to enable:
  • Fault tolerance (no single point of failure)
  • Low latency/performance (serve users from the nearest region)
• Design patterns: Reusable practices for common problems.
  • Example: publisher-subscriber helps notify millions of users without overloading services.

Live streaming product requirements (feature-to-technical mapping)

The tutorial frames requirements like a typical PRD process and then converts them into engineering constructs.

Product requirements (examples)

• Watch live video (primary)
• Comments / reactions
• Graceful degradation (lower quality under bandwidth limits)
• Multiple device support
• Optional examples: ads, disclaimers/news flashes, banners, etc.

Engineering workflow emphasized

1. Prioritize user-visible features.
2. Convert features into data definitions (objects/tables such as videos, comments, likes).
3. Define APIs/endpoints to manipulate/query data.
4. Add engineering requirements:
   • Fault tolerance / avoid system collapse
   • Extensibility (avoid tightly coupled code; anticipate requirement changes)
   • Testing (cover edge cases plus load testing/capacity estimation before deep implementation)

System design example: live streaming backend (high-level blueprint)

The course walks through a backend decomposition and an API/data modeling approach.

Server-side API concepts

• Video retrieval tailored to streaming:
  • Conceptually like get video, but returns video frames/segments rather than the whole video.
  • Uses device type and playback offsets to decide what to return (e.g., “send the next 10 seconds for this device”).
• REST-like examples:
  • GET for retrieving frames (with resolution/device/offset)
  • POST for comments (client doesn’t need device/offset for persisting comments)
• Naming clarification:
  • “get video” might be better renamed to “get video frame”.

Database modeling (example tradeoffs)

• Comments:
  • Suggested as an SQL schema (tables for comments, users, videos, with foreign keys).
• Video storage:
  • Suggested as file system / object storage rather than a relational DB for static video files:
    • file system idea: HDFS
    • object storage: S3 / SDFS
• Comments might also be modeled with NoSQL in concept:
  • described as key-value-ish when relational joins/transactions aren’t needed.
• Main idea:
  • Choose storage based on whether data is static media vs frequently changing structured interactions.

Protocol choices (client-server interaction)

• Comments:
  • HTTP/REST style is sufficient since comment actions aren’t continuous.
  • Statelessness helps: all required context is included in the request.
• Live video streaming:
  • HTTP can be used, but the course suggests specialized real-time protocols.
  • WebRTC is discussed as a peer-to-peer protocol suitable for real-time video transport.
  • Reliability vs real-time efficiency:
    • If current/live state matters → tolerate dropping earlier frames
    • If ordered/reliable context matters (e.g., “lecture context”) → prefer reliability
• Streaming standards referenced:
  • DASH (Dynamic Adaptive Streaming over HTTP)
  • HLS (HTTP Live Streaming)
  • Both adapt quality based on bandwidth/network conditions.

Reliability/performance components

• Use CDNs for performance and fault tolerance (offload static/buffered media from origin servers).
• Mention authentication concerns when using CDNs.
• Reference tooling patterns: load balancers, message queues, caching, and leveraging cloud-provided solutions.

Video ingestion → transformation pipeline (MapReduce + segmentation)

A central section explains how to make raw video viewable across devices.

• Raw 8K cannot be served directly to all clients.
• Transform the raw stream into multiple resolutions (1080p, 720p, 480p, possibly 144p) and formats (e.g., H.264, with proprietary formats mentioned).
• Proposed pipeline:
  • Split raw video into 10-second segments
  • Process segments in parallel into different resolutions/formats using a MapReduce-like design pattern
    • map tasks per target output
  • Store processed outputs back into storage (e.g., S3-like services)

From high-level design to low-level design (code-focused methodology)

The course transitions into low-level design (LLD).

• High-level design: components and interactions (APIs/network calls).
• Low-level design: implementation details (coding component behavior), possibly trading breadth for depth.

Low-level design driven by user actions (use cases → classes)

• Emphasis: start from user use cases instead of only OO class hierarchies.
• Use cases for a consumer viewing video:
  • Play video at timestamp
  • Resume (“watch from where I left off”)
  • Continuous playback with buffering
  • Select maximum quality allowed by device/network
• Introduces modeling artifacts:
  • Use case diagrams
  • Class diagrams
  • Sequence diagrams / timelines (time-ordered request flow)

Sequence flow example (timeline)

• User requests playback:
  • user sends a request to a video consuming service to get playback position (seek timestamp)
  • then fetches frames repeatedly for continuous playback
• Notes overhead concerns:
  • extra calls (e.g., authentication flow) may add complexity and may be optimized.

Concrete coding example (language-agnostic)

Implements core objects such as:

• Video (id, frames, metadata)
• User (id/name/email)
• WatchedVideo (video id, user id, seek timestamp)
• VideoConsumingService (or similar API logic: fetch seek time)
• FileSystem (returns frame for video id + timestamp)

Validation concepts covered:

• return the correct frame for the requested timestamp
• handle invalid timestamps via exceptions vs returning blank results

Where this tutorial directs viewers to go next

• Recommends the creator’s:
  • free “Interview Ready” section for system design fundamentals and patterns (load balancing, rate limiting, charts, scaling)
  • paid section for more advanced system design

---

Main speakers / sources

• Gaurav Sen (course creator; primary speaker throughout)
• References/examples from well-known services/standards, including:
  • Google Maps, YouTube, Hotstar, Zoom, Netflix
  • CDNs
  • DASH/HLS, WebRTC
  • H.264