How to backtest a strategy properly

01What a backtest can and cannot tell you

A backtest answers one narrow question: did these exact rules, with these exact settings and cost assumptions, make or lose money on this particular stretch of history? It cannot tell you the future. Its best use is rejecting ideas: if a rule cannot survive a careful historical test, there is little reason to risk money on it.

Treat a good result as "not yet disproven", never as proof.

02Split your data: in-sample and out-of-sample

The most important habit is to separate the data you build on from the data you judge on.

• In-sample data is where you develop the idea and choose settings.
• Out-of-sample data is held back. You run the finished strategy over it once, with no further changes.

If you keep adjusting the strategy after peeking at the out-of-sample result, that data has quietly become in-sample, and the test no longer means much. Decide your acceptance criteria before you look, for example a maximum drawdown you can tolerate and a minimum number of trades.

03Walk-forward analysis

Walk-forward testing automates the in-sample and out-of-sample idea repeatedly. You optimize on one window of history, test the chosen settings on the next window that follows, then slide both windows forward and repeat. Stitching the out-of-sample segments together gives a result that never relied on knowing the future settings.

NinjaTrader's Strategy Analyzer offers walk-forward optimization, so you can run this inside the platform. It is slower than a single backtest and it still cannot rescue a strategy with no real edge, but it is a much stricter test than one optimized run.

04Costs: commissions and slippage

Many backtests look good only because costs are left out. Include them from the first run.

• Commissions and fees per contract, per side, using what your broker and exchange actually charge you.
• Slippage, the gap between the price you expected and the price you got. Market orders and stop orders usually slip; a limit order may not fill at all. A strategy that trades often is far more sensitive to these costs than one that trades rarely.

A useful check is to double your slippage assumption and see whether the strategy survives. If a small cost change turns it negative, the edge was thinner than it looked.

05Enough trades to mean something

A handful of winning trades is an anecdote. Small samples are dominated by luck, and a short run of results can look excellent or terrible by chance. There is no single magic number, but more independent trades give you more confidence, and a strategy that only trades a few times a year needs a long history to judge. Also check that results are not concentrated in one lucky period or a few outsized trades.

06Avoid look-ahead bias and other traps

• Look-ahead bias: using information that was not available at the time of the decision, such as the close of a bar to make a decision that is filled at that bar's open, or a daily high known only after the day ended.
• Survivorship and data quality: gaps, bad ticks and rolled futures contracts can distort results. Know how your data is stitched.
• Bar-based fill assumptions: if your data is only minute bars, the order of events inside a bar is guessed. When stop and target both sit inside one bar, the backtest cannot know which was hit first. Higher-resolution data reduces the guesswork.
• Too many tweaks: every change after seeing results increases the risk of overfitting.

When a backtest looks acceptable, the next step is not real money but forward testing.

FAQFrequently asked questions