Video summary

Global AppSec Dublin: [T]OTPs Are Not As Secure As You Might Believe - Santiago Kantorowicz

Main summary

Key takeaways

Educational

Main ideas / lessons conveyed

  • OTP/TOTP are often low-entropy challenges (typically too-short numeric codes), making them vulnerable to guessing/brute-force—especially when implementations allow:
    • long validity windows,
    • multiple codes being valid at once,
    • repeated attempts,
    • and/or parallel targeting of many users.
  • “Time window” math matters:
    • OTP (e.g., SMS/email codes) is typically valid for a few minutes (a fixed window).
    • TOTP (authenticator-app codes) is generated using time + secret and typically has a sliding window where multiple codes from adjacent time steps can be valid.
  • Real-world attack feasibility depends heavily on product/UX decisions:
    • how many attempts a user can make,
    • how many OTPs can be requested within the validity time,
    • and whether the system “reveals” failures (or stays silent until the end).
  • Common implementation bugs can further weaken security, sometimes turning “one-time” into “reusable” in practice (e.g., wrong session binding, client-side validation, legacy login flows without 2FA).
  • Mitigations rarely solve the problem alone; best results come from combining controls (rate limiting, better entropy, dynamic defenses, careful messaging, and potentially stronger factors).

Concepts and terminology recap (as presented)

  • OTP (One-Time Password)

    • Example described: Uber sends a 4-digit SMS OTP (digits 0–9).
    • Usually:
      • randomly generated,
      • expires after a few minutes,
      • often allowed multiple attempts during that time.

  • TOTP (Time-based One-Time Password)

    • Not purely random: generated by algorithm(time, secret).
    • Typically:
      • 6–8 digits,
      • expires after a few minutes,
      • due to time-step alignment, there can be a window where more than one TOTP is valid.

  • Key security observation

    • Not only “one code at a time” may be valid—implementations can allow multiple concurrent valid codes, which changes brute-force probability.

Attack scenario / story motivating the talk

  • Research context at Twilio (company division working on OTP/TOTP products).
  • A bug bounty report claimed: with unlimited attempts, an attacker could add an device using OTP-based verification.
  • Initial reasoning suggested success chance was small (e.g., “one chance in a million” per guess), but the presenter later found the math/behavior of validity windows and repeated requests increases success probability far more than expected.
  • Outcome:
    • they wrote a blog post and delivered this talk,
    • received recognition that it was a valid finding.

Core methodology: probability of guessing within OTP validity windows

Parameters used repeatedly in the talk (example defaults)

  • OTP length / space

    • 4-digit numeric → (10^4 = 10,000) possibilities
    • 6-digit numeric → (10^6 = 1,000,000) possibilities
    • 8-digit numeric → implied accordingly (larger space)

  • Attempts allowed per window

    • typical example uses 5 attempts (noted that many sites allow ~3–5; some allow more)

  • Validity time / windows

    • OTP valid for ~3 minutes (because SMS/email delivery delays)
    • within that 3-minute window, the system may allow multiple OTP requests (example limit: up to 10 valid OTPs at a time)

  • Sliding window behavior for TOTP

    • typical time step is 30 seconds
    • commonly valid across two time steps (~1 minute total validity window)

Single-user guessing (OTP via SMS/email)

  • 4-digit OTP example

    • 10,000 possibilities → 1/10,000 per guess
    • allowing 5 attempts within 3 minutes increases success chance (presenter gives ~0.05% for 5 guesses, then later much higher when multiple OTPs are valid concurrently)
    • with up to 10 OTPs valid in one window, success probability becomes ~0.5% (~1 in 200) for that scenario

  • Repeating across multiple windows

    • the presenter corrects a misconception: even though OTPs expire, the attacker can request more codes in the next window, and the cumulative probability over time becomes large.

  • Conclusion from examples

    • Numeric OTPs that are too short + enough attempts/windows lead to high success probabilities over days/months.

“One user” over longer periods

  • 6-digit OTP
    • presenter’s computed success probability:
      • about 2.3% per day for the given assumptions
      • about 91% over 100 days
      • about 99% over ~6 months
  • Presenter claims these probabilities are not “made up”:
    • they built a simulator to validate calculations,
    • simulator results matched the derived numbers (e.g., 91, 2.3, ~99).

Why TOTP makes it worse (compared to naive expectations)

  • For TOTP, the valid codes aren’t strictly isolated to one exact 30-second bucket; because of implementation windows:
    • multiple TOTP values can be valid in overlapping time steps,
    • brute forcing must account for how attempts distribute between time steps.
  • Presented outcome:
    • TOTP can be more favorable to attackers if the system uses permissive windowing and allows sufficient attempts/requests.

Parallelizing the attack (credential stuffing + OTP guessing)

Why attacking many users at once is critical

  • If an attacker already has correct user credentials (from password reuse/credential stuffing), they only need to guess the OTP/TOTP as the second factor.
  • Presenter’s framing:
    • If OTP is a single step that grants access once guessed correctly, then attacking in parallel across many accounts dramatically increases total chances.

Multi-user success examples (as presented)

  • 6-digit OTP

    • Attack 100 users in parallel → presenter suggests ~91% chance of success in one day (under their assumptions).

  • 8-digit OTP

    • Even with fewer digits expanded to 8-digit, the presenter claims parallel attacks still allow high success probability over feasible time horizons (e.g., “in 100 days” style outcomes).

  • Rate limiting alone doesn’t fully solve it

    • Example rate limit used: 60 OTPs/month (i.e., ~2/day)
    • Presenter claims that with sufficiently parallel targeting, attackers still achieve high probability of at least one success within about a month (for large user counts like 1,000+ users).

Alphanumeric codes: partial improvement, but tradeoffs

  • Presenter argues that moving from numeric-only to alphanumeric increases the search space and reduces success probability.
  • However:
    • it may cause user friction and raise support/UX issues.
  • They explicitly caution:
    • “Don’t just switch to alphanumeric OTPs” and assume it fixes everything.

Implementation/design pitfalls highlighted (beyond brute-force math)

The talk includes examples of common engineering mistakes that can undermine OTP security even further, such as:

  • Flow bypass / insufficient session binding
    • System checks the OTP is valid but doesn’t bind it to:
      • the correct user, or
      • the correct session/context.
  • Client-side comparisons
    • Putting OTP checks on the client (or storing expected OTP as hidden client-side values).
  • Legacy login routes without 2FA
    • New login requires 2FA, but older login endpoints still exist and don’t require 2FA.
  • Reusing OTPs
    • OTPs that should expire or be single-use still work when reused.
  • Insufficient entropy / digit length
    • Examples include too-short OTPs (e.g., customer wanting 3 digits) leading to “very risky” outcomes.

Mitigations proposed (with actionable bullet points)

Presenter states there is no single mitigation that fully works, but suggests layered defenses:

1) Cryptographic correctness / consistency in delivery

  • Deliver the same OTP for the same validity window
    • Particularly for SMS/email delivery (avoid “rotating” multiple times unpredictably in a way attackers can exploit).
  • Encrypt OTP generation/delivery state server-side
    • Example idea: protect secrets using HMAC or equivalent.
    • Ensure the server must verify against protected values, not client-side checks.
  • Regenerate OTP only when genuinely required
    • Create a new OTP when a new request is made, but keep consistency rules well-defined.

2) Rate limiting and attempt controls (granular)

  • Limit attempts per user
    • Per day / per week / per month granularity.
  • Avoid extremely permissive “global” limits
    • If you allow too much, attackers can parallelize.
  • Use incremental failure delays
    • After failed attempts, introduce progressive delays rather than allowing many rapid tries.
  • Reset attempt counters on successful login
    • Reduce lock-in friction for real users.

3) Detect automated abuse

  • IP-based and endpoint rate limits
    • Attack traffic often concentrates on the same endpoints from fewer IPs unless attackers use large bot infrastructure.
  • Detect repeated failures / patterns
    • Monitor unusual OTP request volume and failure rates.

4) Increase entropy dynamically or adapt under attack

  • Dynamically increase OTP size
    • e.g., escalate from 6 digits numeric to more characters when suspicious activity is detected.
  • Switch from numeric to alphanumeric under attack
  • Add step-up challenges
    • Captchas (not great UX, but can help during active attack).
    • Knowledge questions (sucks UX, but can help).
  • Multi-channel verification
    • If attacked, send OTP via another channel:
      • SMS → email OTP, or email link with much higher entropy than digits.
    • Keep user messaging non-alarming while still secure.

5) Don’t leak which part was wrong

  • Avoid disclosing whether OTP or password is wrong during the flow
    • Prevent attackers from learning information needed to optimize credential stuffing.
    • (Idea repeated in Q&A: don’t reveal intermediate failure; answer after full input.)

6) Strengthen authentication factors beyond OTP (strategic options)

  • U2F / hardware security keys
    • Stronger, but may not be feasible due to user adoption/churn.
  • Push authentication
    • Alternative factor with different UX and risk properties.
  • Direct carrier billing / “recovery billing”
    • Mentioned as a mechanism to charge/authenticate via mobile carrier billing (presented as low friction in supported countries, but not guaranteed secure; the presenter didn’t verify deeply).
  • Service-level policy
    • Consider step-up to stronger factors for high-risk users (e.g., high-value accounts).

7) Security validation and sizing the risk

  • Do the math for your own parameters
    • Assumptions change with:
      • window size,
      • number of OTPs valid at once,
      • allowed attempts,
      • and concurrency/rate limiting.

Q&A highlights (additional constraints / clarifications)

  • “If you don’t disclose whether OTP/password is wrong, is credential stuffing harder?”
    • Presenter agrees it helps, but suggests user/UX and risk tradeoffs.
  • “Does reducing attempts from 5 to 3 improve much?”
    • Presenter claims limited impact in their model; attackers succeed over time mostly due to windows and concurrency.
  • “Have you tested on real systems?”
    • Presenter did not provide a full comparative survey of Google/Microsoft/etc. settings, and says behaviors vary; attackers can exploit many real systems with permissive behaviors.
  • “What about backup codes?”
    • Presenter didn’t provide math for backup codes; notes backups often exist and must be designed with the same risk mindset. In many systems backups still lead back to SMS OTP risk.

Speaker/source list

  • Santiago Kantorowicz (presenter; works at Twilio; security-related role connected to OTP/TOTP products)
  • Audience participants / questioners (unnamed; asked multiple questions during the session)
  • Uber (example OTP usage and screenshot reference)
  • Authy, Google Authenticator, RSA token app (examples of TOTP/token usage)
  • Citibank (example mentioned re: OTP digit-length change)
  • Bug bounty researcher / report author (referenced as the source of the initial unlimited-attempts finding; not named)

Original video