Video summary

Your Backtest is Wrong - How to do it Properly

Main summary

Key takeaways

Finance

Core message

The video argues that many trading strategies “look amazing” in backtests but fail live because backtests are often done incorrectly. It highlights that, without proper methodology, results can be worse than having no backtest, due to false confidence.

Tickers / assets / instruments mentioned

  • None explicitly in the provided subtitles (no stocks, ETFs, bonds, commodities, forex pairs, or crypto tickers).

Key numbers / figures called out

  • $169.5 vs $170 to illustrate slippage of $0.5.
  • Warning about unrealistic backtests: comparing an equity curve with 0 slippage / 0 commissions versus realistic execution costs (no specific numeric return provided).
  • Backtest length guidance:
    • Single month: ~20 trading days (too few trades → “noise”).
    • Recommended: at least 5 years, ideally more.
  • Overfitting example:
    • 90K (90,000) parameter combinations tested for an “opening range breakout” variant (suggesting a heavy search can produce lucky fits).
  • Example data split timeframe:
    • Train: 2016–2021
    • Validation: 2022–2026
  • Win-rate / losing streak discussion:
    • Example win rate: 60%
    • Losing streaks: two or three losers in a row can be normal; five to ten losers can still happen.

Proper backtesting rules (4-part checklist)

1) Include trading costs

  • Model commissions (fees per trade entry and exit).
  • Model slippage (difference between the intended and executed price due to volatility and liquidity).
  • If a backtest ignores slippage/commissions, the result should be discarded.

2) Use a long enough time period

  • A single month (~20 trading days) is not enough.
  • Markets switch between regimes (e.g., trend vs. chop; calm vs. violent), so short results can be misleading.
  • Require ≥ 5 years, ideally more.
  • If backtests shown publicly cover < 1 year, the video urges skepticism.

3) Prevent overfitting with a train/validation approach

  • Split data into:
    • Train data: used to choose the best parameters
    • Validation data: used to test generalization
  • Only keep strategies whose results still hold on validation.
  • Explicit caution:
    • If validation fails, do not keep tuning until validation looks good—otherwise, validation becomes another training set and the test is invalidated.

4) Run a Monte Carlo simulation

  • From backtest/trade history, estimate:
    • win rate
    • distribution/size of wins and losses
  • Generate tens of thousands of randomized trade sequences consistent with those probabilities.
  • Analyze the distribution of outcomes, including:
    • worst losing streaks
    • deepest drawdowns
    • probability of account wipeout
  • Interpretation rule:
    • If most simulated scenarios survive and remain profitable, the edge is more credible.
    • If many scenarios blow up, the original backtest likely benefited from luck (a “lucky run”).

Explicit recommendations / cautions

  • Throw out backtests that:
    • ignore slippage and commissions
    • use overly short periods (e.g., ~1 month or <1 year)
    • do not control for overfitting
    • do not include a Monte Carlo robustness check
  • Do not confuse validation-tuning with proper validation.
  • Before trusting a “trading guru” backtest, verify:
    • ≥ 5 years of coverage
    • execution costs are included
    • overfitting controls are used
    • Monte Carlo simulation is performed

Disclosures / disclaimers

  • No explicit “not financial advice” disclaimer was present in the provided subtitles.

Presenters / sources mentioned

  • “Trading guru named Scarface”: criticized for showing a profitable backtest without considering trading costs.
  • “TJ R”: mentioned posting live results from a past 6 months, using Trade Zella.
  • TradeZella: software/tool referenced for tracking data.

Original video