Video summary

I Did it! My BEST AI Trading Bot vs. ChatGPT | Here's What Happened

Main summary

Key takeaways

Technology

Summary of technological concepts, product features, and analysis (auto-subtitles)

AI trading-bot vs. ChatGPT “challenge” setup

  • The creator compares their own trading strategy (called “stiff surge / champion strategy”) against a ChatGPT-generated strategy.
  • Fairness constraints:
    • Same markets, same risk rules, same timeframe
    • Same fees and slippage
    • ChatGPT is given 24 hours, while the creator’s strategy had ~5 years of iteration/training time
  • Creator’s strategy claim:
    • 7,192% backtest result (historical test)
    • ~78% up in real trading within the year
  • “Champion” strategy characteristics:
    • Works across multiple coin pairs, with adjustments per market/timeframe
    • Forward-tested for over 2 years
    • Described as the most profitable strategy the creator released

Tools/product: TradingKit + MCP servers + TradingView integration

  • The workflow uses ChatGPT desktop plus MCP servers to give ChatGPT access to trading research/backtesting tooling.

MCP server #1 (free, via tradingkit.com)

  • Provides ChatGPT the ability to backtest strategies automatically.
  • The creator states there’s a community/library with ~159,000 backtests.

MCP server #2 (creator-built; installed from GitHub)

  • Adds a search function for TradingView indicators.
  • Purpose: enable ChatGPT to look beyond basic indicators like RSI and MACD and choose indicator combinations that “make sense.”

Credit/compute model

  • Optimization is performed on TradingKit’s servers, aiming to avoid burning too many ChatGPT credits locally.
  • Creator claims the optimization process “checks in,” so credits are less wasted.

Data/controls provided to ChatGPT (embedded into the prompt)

  • Access to TradingView historical data
  • Access to the creator’s backtesting results
  • Access to a risk management “boilerplate”
  • An optimizer to find best settings
  • All embedded into the prompt (linked in the description)

Prompting and automated strategy generation

  • ChatGPT is prompted as a “top quant strategy developer” to:
    • Autonomously research, build, and backtest
    • Generate new strategies and optimize every 5 minutes
    • Run the loop for 24 hours
    • Attempt to beat a supplied target equity curve (shown via screenshot)

Results: ChatGPT created high-profit strategies but with issues

  • Mid-run update:
    • Many backtests looked promising (including results “over a,000%”)
    • Creator reports ChatGPT can find extreme returns but may also suffer large losses (example mentioned: huge percentage but max drawdown ~68% on full data)
    • Concern: ChatGPT may “go into a rabbit hole,” over-optimizing (“throwing ideas at the wall”) to maximize net profit rather than robust performance
  • Final ChatGPT candidate:
    • Claimed ~8,110% profit with max drawdown ~31%
    • Reported stats:
      • 136 closed trades
      • profit factor ~6.24
    • Creator’s notes:
      • This beats the creator’s strategy in the original comparison backtest metric
      • But the challenger strategy appeared to work mainly on BTC
      • In tests of top ~20 coin pairs, it allegedly only meaningfully worked on BTC

Side-by-side comparison vs. creator’s bot

  • Drawdown:
    • Creator max drawdown: ~17.96%
    • Challenger max drawdown: ~31%
  • Number of trades:
    • Creator: ~491 closed trades
    • Challenger: 136 trades
  • Win rate:
    • Creator: ~52%
    • Challenger: ~38%
  • Trade direction difference:
    • Creator strategy includes long and short trades (helpful in bearish markets)
    • Challenger strategy appears long-only, potentially trading during bearish conditions and lowering win rate

Two “fairness/quality” criticisms of the ChatGPT strategy

  1. Trailing stop-loss implementation

    • Challenger uses a trailing stop-loss on TradingView.
    • Creator argues this adds operational risk because TradingView trailing stops rely on alerts/execution and could cause incorrect exit timing versus broker-side, one-command order placement.
    • Creator prefers specifying entry, take-profit, and stop-loss in a single broker command to ensure execution at exact prices.

  2. Backtest horizon limitation / overfitting concern

    • Creator claims ChatGPT only backtested up to ~2023, not the full available history.
    • They argue this can cause overfitting (great results on limited optimization period, failure on longer/unseen data).
    • Creator says their full historical backtest caused ChatGPT’s strategy to get “wrecked” during additional periods.

Overall conclusion of the experiment

  • Backtest conclusion: ChatGPT “technically lost” versus the creator’s bot when evaluated on broader data (implied to correct for overfitting).
  • Proposed next step: a real-money forward test for 7 days to see whether the challenger can beat the strategy live.
  • Creator’s stance on AI:
    • Trust AI for finding strategies
    • Currently do not let AI trade automatically
    • Best practice suggested: use AI to monitor rules (e.g., pause a bot) rather than fully delegating execution

Guidance/tutorial-style rules for building/testing AI trading strategies

The creator provides explicit rules to avoid misleading results:

  • Don’t treat black-box AI trading systems as trustworthy
    • Some products can fake equity curves (they reference “clawude design” style HTML equity-curve pages).
  • Backtest realism requirements
    • Use enough trades: roughly 100+ trades (more depending on timeframe)
    • Include fees and slippage, and overestimate them to avoid hidden losses
  • Diversify
    • Don’t rely on a single bot
    • Use different strategies/bots/timeframes/pairs to smooth portfolio equity
  • Forward test
    • Even if backtests look fantastic, most strategies fail out-of-sample
    • Creator claims prior extensive testing (2,000+ strategies)
  • Execution detail reminder
    • Ensure stops/entries behave correctly in real trading
    • Emphasize broker-side order control versus TradingView trailing stop alert reliance

Main speakers / sources

  • Main speaker: The video creator/trading strategy author (speaks throughout; not named in the subtitles)
  • Referenced tools/platforms: TradingKit / tradingkit.com, ChatGPT desktop, MCP servers, TradingView (indicators, alerts, historical data), and an external community/backtest library
  • Mentioned models/entities: ChatGPT and Claude (as comparisons/benchmarks; the described experiment is primarily ChatGPT vs. the creator’s bot)

Original video