Backtesting a Strategy Without Fooling Yourself

Backtesting is the process of running a trading strategy's exact rules against historical price data to see how it would have performed. Done honestly, it tells you whether an idea has any real statistical edge before you risk money on it. Done carelessly, it will tell you that almost any idea is a winner, which is the trap that catches most people who build their first system.

What is backtesting?

A backtest takes a set of rules (when to enter, when to exit, how big to size the position) and applies them mechanically to historical data, bar by bar or tick by tick. Each time the conditions are met, the simulation records a trade. String enough of those trades together and you get an equity curve, a drawdown history, a win rate, and an average trade size.

The data window matters more than most people think. A strategy tested only on 2019 saw a quiet, low-volatility market. One tested through 2008 or the spring of 2020 saw something entirely different. If your data doesn't include a real stress period, your backtest hasn't actually been tested yet.

Why every strategy works in a backtest

Give yourself enough adjustable parameters, moving average lengths, stop distances, entry filters, and you can fit almost any rule set to almost any historical data. This is curve-fitting: the strategy isn't finding a real market pattern, it's memorizing the noise in the specific data you handed it. The equity curve looks beautiful because you kept tuning knobs until it did.

Two quieter problems do the same damage. Look-ahead bias creeps in when a backtest uses information that wouldn't have been available at the moment of the trade, like deciding to buy at the day's open based on where the day's close ended up. Survivorship bias shows up when you test only on symbols or index constituents that still exist today, which leaves out every instrument that already failed. Both make a strategy look better than it ever could have traded in real time.

A backtest that hasn't survived out-of-sample data is just a well-decorated story about the past.

The real costs that a clean backtest ignores

A backtest that fills every order at the exact theoretical price is describing a market that doesn't exist. One ES tick is 0.25 points, worth $12.50 per contract. In a fast market, a stop order can realistically slip two or three ticks past where you expected the fill, and that's before commission and exchange fees on both sides of the trade.

Costs matter most for strategies that trade often. A system that nets $15 per trade before costs and pays $5 in commission and slippage still has an edge, but a system netting $8 before costs is now losing money the moment it goes live. Run the backtest again with realistic slippage and commission built into every fill, not added as an afterthought at the end, and watch how many strategies stop looking interesting.

What is out-of-sample testing?

Out-of-sample testing means splitting your historical data into two parts before you touch either one. You build and tune the strategy on the first part (in-sample) and then, only once, run it unchanged against the second part (out-of-sample) that it has never seen. If the results hold up, you have some evidence of a real edge rather than a well-fitted story.

Walk-forward testing extends the idea across the whole dataset: optimize on a window, test on the next unseen window, roll forward, and repeat. It's slower and less flattering than a single all-data backtest, but it's a much closer approximation of what actually happens when a strategy meets a market it has never traded before.

Signs the backtest is honest, not decorated

Pull the ten best trades out of the results and look at what's left. If the strategy only works because of a handful of outlier trades, that's fragility, not edge. Check the strategy across several distinct years and market regimes rather than one smooth stretch, and count how many free parameters it actually has: the more knobs you tuned to get here, the less you should trust the result.

Paper trading in real time, with live data and no hindsight, is the next honest checkpoint after a backtest looks solid. It won't catch curve-fitting on its own, but it will catch execution assumptions a backtest can't see: real order queues, real latency, real fills. A strategy that survives out-of-sample data, realistic costs, and a stretch of paper trading has earned a little more trust than one that has only ever seen a spreadsheet.