Overfitting: when a strategy memorises the past

01What curve fitting means

Any historical data contains two things: a possible repeatable pattern, and random noise that will never repeat in the same way. A strategy with enough adjustable settings can be tuned until it fits both. The result looks wonderful on that data because it has effectively memorised it.

The more parameters a strategy has, and the more combinations you try, the easier it is to find a combination that worked by luck alone. Trying thousands of settings and keeping the best one is guaranteed to produce an impressive backtest, even on random data.

02Warning signs

• Many parameters for a simple idea, especially ones with odd precise values such as a 37-period setting with a 0.83 multiplier.
• A very narrow peak: results are excellent at one setting and poor at the neighbouring ones.
• Special-case rules added to dodge specific past losing trades.
• Great in-sample, weak out-of-sample results. See backtesting.
• No reason for it to work. If you cannot explain in plain words why the behaviour might exist, be sceptical.
• Few trades behind a smooth equity curve.

03Robustness checks you can run

Parameter stability

Change each setting a little in both directions and look at the surrounding results. A robust idea shows a broad plateau where many nearby settings are acceptable. A single isolated spike is a red flag. Choose settings from the middle of a plateau, not the top of a spike.

Different markets and periods

If the logic reflects real market behaviour, it often shows at least some life on related instruments or in other years, even if less strong. If it only works on one contract in one year, it is likely fitted.

Out-of-sample and walk-forward

The strict versions of the checks in the backtesting guide are the strongest defence, because they test on data that played no part in the design.

Monte Carlo, at a conceptual level

A Monte Carlo test reshuffles or resamples your list of trades many times to create alternative orderings. The path of a real equity curve depends partly on the order in which wins and losses happened. By seeing the range of drawdowns across many reshuffles, you get a feel for how bad a normal unlucky streak could be. It cannot add information that is not in the data, but it is a sobering reminder that your historical drawdown is one sample, not a ceiling.

Cost stress

Raise commissions and slippage and see whether the strategy survives. Fitted strategies are often fragile here.

04Prevention is simpler than diagnosis

• Start from a clear market idea, then test it, rather than searching for patterns.
• Keep the number of parameters small.
• Record every variation you try. The number of trials matters when judging how impressive the best one is.
• Reserve out-of-sample data and look at it once.
• Prefer settings that are neighbours of other good settings.

Even with all of these, no check removes the risk. After the tests, the honest next stage is forward testing on live data you could not have fitted. Past performance does not guarantee future results.

FAQFrequently asked questions