ZenTrader

Free tool

Is your backtest lying to you?

Most strategies look better on paper than they trade. Fifteen questions catch the usual reasons. Answer for the last test you ran.

  1. Costs

    Does every trade pay brokerage, STT, stamp duty, exchange charges and GST?

  2. Costs

    Does it assume some slippage on entries and exits?

  3. Data

    Does the stock list include companies that were later delisted, merged or dropped from indices?

  4. Data

    Are prices adjusted for splits and bonus issues?

  5. Data

    Does each decision use only what was known at that moment?

  6. Execution

    Is the entry price one you could have got?

  7. Execution

    Are stops filled at the open when a stock gaps through them?

  8. Execution

    Is each position small next to the stock's daily traded value?

  9. Portfolio

    Is it tested as one account, with limited cash and positions that overlap?

  10. Robustness

    Are there 100 trades or more?

  11. Robustness

    Was part of the history kept back and tested only once, at the end?

  12. Robustness

    Do the results hold up when each setting is nudged up or down?

  13. Robustness

    Does it cover a crash and a long sideways stretch, not only a bull run?

  14. Robustness

    Is it still profitable without its five best trades?

  15. You

    Could you sit through its worst drawdown and longest losing streak?

15 questions to go.

The questions are weighted. Missing costs, look-ahead and survivorship count three times as much as a small sample. Read why most backtests lie for the longer version.

Common questions

What is survivorship bias in backtesting?

Survivorship bias comes from testing only on stocks that exist today. Companies that collapsed, merged or were delisted are missing, and those are often the trades that would have lost money, so the backtest looks better than reality.

What is look-ahead bias?

Look-ahead bias is a backtest using information that was not available at the moment of the decision. Two common cases are buying at the open on a signal that needs that day's close, and using today's index members to pick stocks for past years.

How many trades does a backtest need?

Enough that luck cannot decide the result. Twenty trades can show a profit for a losing strategy. A hundred or more, across a rising market, a crash and a sideways stretch, is a sensible minimum.