Video summary

Stop Devs Burning Your AWS Bill - Claude Code Budget Control with LiteLLM

Main summary

Key takeaways

Technology

Tech concept / solution overview

  • Goal: prevent per-developer AWS cost overruns when running Claude Code via AWS Bedrock.
  • Earlier concept: three ways to enforce per-developer budgets (not covered in detail here).
  • This video implements the first approach: a Light LLM gateway/proxy running locally in Docker that sits between Claude Code and Bedrock.

Architecture (how budget control works)

Without the gateway

  • Claude Code calls Bedrock directly.
  • All users share the same IAM credentials, so all costs aggregate into one CloudWatch figure.
  • Result: no per-developer visibility/enforcement.

With the Light LLM proxy

  • Claude Code is configured to call the proxy on localhost (port 4000) instead of Bedrock.
  • Light LLM creates virtual keys per developer, where each key has:
    • a max budget cap (e.g., daily)
    • a budget reset/duration window (e.g., every 24 hours)
    • rate limits, including:
      • TPM (tokens per minute)
      • RPM (requests per minute)

When a request arrives, Light LLM:

  1. checks remaining budget and rate limit state
  2. if limits are exceeded, returns an error immediately
  3. does not call Bedrock, so no AWS cost accrues

Key behavioral “rules” emphasized

  1. Enforcement is primitive (preventative, not reactive)

    • The proxy blocks before spending happens.
    • It’s not cleaning up after the fact.

  2. Rate limiting and budget caps are independent

    • Rate limits prevent runaway behavior regardless of cost.
    • Budget caps stop when cumulative spend crosses a threshold for governance.

  3. Spend tracking is automatic across models

    • Light LLM maintains a model cost map and updates it from the Light LLM GitHub on a schedule.
    • The user doesn’t implement cost calculation logic.

Build / tutorial steps (implementation details)

Config file (config.yml)

  • Creates config.yml for Light LLM, including:
    • model name / Bedrock model mapping
    • AWS credentials and region sourced from environment variables
    • a master/admin key used only to create virtual keys (not for LLM requests)

Docker Compose setup

  • Runs the Light LLM proxy on port 4000
  • Runs Postgres for persistent budget state
  • Uses depends_on + healthcheck so Light LLM won’t crash until Postgres is ready

Startup & login

  • Open localhost:4000 in a browser
  • Log in with:
    • username: admin (default)
    • password: the configured value

Virtual keys created (demo parameters)

Developer keys are created via API (curl) rather than UI to mirror scripting/team onboarding:

  • Dev Alice (working example)

    • used to show real-time spend tracking

  • Dev Bob (demo)

    • max budget: ~$5
    • budget duration: 24 hours
    • intentionally low/rapid budget test to trigger enforcement quickly

  • Dev Charlie

    • higher budget: example ~$50
    • budget duration: 24 hours
    • demonstrates that the model can be set per configuration

Claude Code integration (minimal client changes)

Claude Code is configured similarly to how it would be for Anthropic:

  • Set Anthropic base URL to localhost:4000
  • Use the Light LLM virtual key as the “Anthropic API key”

Important effects:

  • Developers don’t need AWS credentials locally
  • Light LLM validates the key and uses the proxy’s AWS credentials to call Bedrock

Live demonstration (budget enforcement in action)

Dev Alice

  • Spend is tracked in real time.
  • Daily budget counters update.
  • Reset happens on the 24-hour schedule.

Dev Bob (very low budget test)

  • A second request fails.
  • Light LLM returns an error message such as “budget exceeded” including:
    • key identifier
    • current spend
    • max budget
  • The error occurs before any request reaches Bedrock → no further cost accrues
  • In the dashboard:
    • Dev Bob is stuck at/near the cap
    • further requests are blocked until the window resets
  • Optional admin capability:
    • manual reset of spend in the dashboard
    • developers cannot manually reset

What’s explicitly not handled (limitations / next steps)

  • Not covered:
    • developer onboarding at scale
    • key rotation
    • high availability for larger teams (e.g., 20+ people)
  • Mentioned follow-up:
    • later video moves the same proxy pattern to ECS for production teams
  • Series continues with another approach:
    • a native AWS approach using:
      • Bedrock model invocation logging
      • CloudWatch alarms
      • a Lambda function that cuts off IAM access when thresholds are hit
    • No third-party proxy in that approach

Links / resources referenced

  • Notes that all config files are linked in the description.
  • Mentions AWS Builder Center published a reference architecture for the same gateway/interception pattern (link to be added in description).

Main speakers / sources

  • Speaker: likely the series creator/host “Joshua/author” (not named in the subtitles)
  • Primary technical source referenced: AWS Builder Center reference architecture
  • Product/library source referenced: Light LLM, including its GitHub for model cost map updates

Original video