Video summary

Rate Limiter Pattern in Microservices | Resilience4j RateLimiter Explained

Main summary

Key takeaways

Educational

Main ideas / lessons

  • Problem: too many requests (traffic spikes)

    • If a downstream service (e.g., the inventory service) supports only limited throughput (e.g., 100 requests/sec), but suddenly receives far more (e.g., 10,000 requests/sec), it can:
      • Exhaust threads
      • Spike CPU usage (can reach near 100%)
      • Exhaust database connections
      • Slow down and potentially crash
    • If one service crashes, calling services can also fail, potentially taking down the broader microservice system.

  • What a rate limiter is

    • A rate limiter controls how many requests are allowed within a specific time period.
    • Example:
      • If the limit is 5 requests per second:
        • Requests 1–5 are allowed
        • The 6th request (and beyond within that window) is rejected with “too many requests”
        • The quota refreshes next second

  • Why rate limiting is needed

    • Traffic spikes that overload services
    • Attackers / DDoS-style request floods
    • Misbehaving clients sending repeated/random requests
    • Protecting expensive APIs (limit per client/service tier)
    • Supporting plans (e.g., basic vs premium, different allowed request rates)

  • Where rate limiting sits in microservice architecture

    • Rate limiting can be applied at multiple points:
      1. At the API Gateway (rejects traffic early to protect the system)
      2. Between services / at the caller side
        • Example: since order service calls inventory service, enforce “don’t call inventory more than X times/sec” from order service
      3. Inside the target service (e.g., within inventory service)
    • Core idea: protect incoming traffic (external) and/or outgoing calls (inter-service).

Implementation methodology (Spring Boot + Resilience4j)

1) Add Resilience4j dependency

  • In pom.xml, add the Resilience4j Spring Boot dependency (version depends on your Spring Boot version).
  • Reload Maven so annotations are available.

2) Add the rate limiter annotation to the call

  • In order service, where it calls inventory service, apply the Resilience4j annotation:
    • @RateLimiter(name = "...")
  • Use a rate limiter instance name aligned with configuration keys (e.g., inventory-service).

3) Define a fallback method for rejected requests

  • If rate limiting rejects a call, handle it via a fallback method:
    • Create a method in the same class
    • Configure the annotation to reference that fallback
  • Behavior described:
    • If fallback returns quantity = 0, the system behaves like it is “out of stock.”

4) Configure rate limiter parameters in application.yml

Under the limiter’s configured name (example referenced: inventory service), set:

  • limit-for-period
    • Max number of allowed calls in a period (example: 2 requests)
  • limit-refresh-period
    • Length of the window/period (example: 10 seconds; quota refreshes every 10s)
  • timeout-duration
    • How long a rejected request is allowed to wait for permission (example: 0 = immediately reject)

Example effect

  • With limit-for-period = 2 and limit-refresh-period = 10 seconds:
    • First two requests in a 10-second window pass
    • Third request is rejected → triggers fallback
    • After 10 seconds, quota refreshes for the next two requests

5) Observe behavior/error and fallback execution

  • When exceeded, the fallback is invoked.
  • The error includes:
    • “request not permitted”
    • “Rate limiter [name] does not permit further calls”
  • The video also mentions a proxy-based approach in Spring Boot:
    • A proxy sits between order service and the annotated method to intercept calls and enforce rate limiting.

Algorithms and library behavior (as discussed)

  • Resilience4j mentions rate limiting algorithms such as:
    • Fixed window
    • Sliding window
    • Token bucket
    • Leaky bucket
  • The key claim in the video:
    • Resilience4j uses a cycle-based rate limiting mechanism (time divided into refresh periods with a fixed number of permissions per period), described as similar to fixed window.

Conceptual guidance: ordering with other resilience patterns

  • The video argues that rate limiting should logically happen before retry:
    • If calls are rate limited, don’t retry—because the system is intentionally rejecting due to rate constraints.
  • It also previews future topics: time limiter, circuit breaker, bulkhead.

Speakers / sources featured

  • Speaker: Not explicitly named in the subtitles (the instructor/presenter)
  • Sources/technologies mentioned:
    • Resilience4j (RateLimiter, fallback, proxy behavior, cycle-based algorithm)
    • Spring Boot
    • Spring Framework Spring Retry (mentioned for contrast with Resilience4j retry)
    • API Gateway
    • Eureka Server (service discovery)
    • Example services: order service, inventory service
    • Other algorithm-related library mentioned: Bucket4j

Original video