London Trading Group

London's Leading Trading Academy

Register now to get access to your free content.

No experience needed. We only show you what works.


Common Mistakes You Make Backtesting Trading Strategies

By 20th June 2019 News No Comments

Algorithmic trading is pretty exciting at the beginning, because if you have some coding and trading experience, in just a few hours, you’ll come up with many trading systems that—at least on paper—are good.

This is rewarding, so you’ll keep doing it.

But if you use your new strategies, you’ll realize that the trades made according to your work have a purely random outcome. Your portfolio moves up and down with an overall downward inclination due to the transaction cost.

Coding a trading strategy is easy, so why doesn’t it make money?

Because selecting the few good ones among the many you come up with in your imagination is very hard.

The important and often underestimated part of an algo trader’s job is to filter out those strategies—those that are already available on academic papers or in your portfolio—rather than discovering new ones.

Because of the flawed method of testing, an infinite number of ideas apparently become profitable.

This is not only true in retail but also in the institutional world. I have never seen a strategy on the marketing brochure of some hedge fund with just average returns in backtesting.

Financial data contains a lot of randomness, so trying to understand ifa strategy made money as a result of luck or because it takes advantage of an actual inefficiency in the market is not a trivial task.

One of the few things that has not changed since I started this job is that I usually do the opposite of what would be natural and intuitive to do, which can often lead to the solution.

To spot a profit opportunity in the financial market, it is necessary to find the reason behind it. This is because the investors behave in such a way as to create this inefficiency in the financial market.

Human nature does not change; therefore, trading opportunity, based on observation of people’s reactions, stand a chance of being profitable in the future.

For example, after a huge drop in value,stocks tend to go up again. This is because investors overreact to bad news by selling. They then realize that the world is not going to end just yet and they buy back what they have just sold for a higher price.

This phenomenon has been studied extensively. Everyone knows it, but investors keep acting irrationally by selling too much and then buying it back.

Acting against our intuition—doing what is emotionally more painful—is absolutely necessary in order to survive in the financial markets.

Going back to the newbie algo trader, he is trying to find the best strategy by testing as many ideas as possible.

The answer is, again, counter intuitive. He should be focusing not on finding new ideas but on how to test them.

The more he tries to create new strategies, the more he is likely to find those that, just by luck, are making money when tested on historical data.

Some of the common backtesting mistakes are obvious and well known.

Everyone knows that when a strategy contains too many parameters, it is more likely to be biased. The temptation is to make it more complicated by introducing even more bias, which makes the system you have worked on for a whole month a profitable one—but only on paper.

Most of the common guidelines on how to run a good backtest are known but ignored, just like good diet habits.

One of the most ignored best practices is to discard systems that are too profitable. To justify this, we don’t need math—just a bit of common sense. The easier way to find this threshold is to have a look at what has happened to the trading systems you coded last year, with a profit factor above three.

Because of misleading advertisements, many people approaching the algo trading world believe that, with a simple trading system, a $500 USD account can become $1,000,000 in just one year. I could explain why this is not possible, but I prefer to think that my readers are educated people who already know this.

One of the hardest parts of the problem to solve while doing a backtest is a particular kind of look-ahead bias that some call the“time-machine bias.”

We found that every time a price pattern occurred, the market goes up.

To code this strategy, we have done everything properly, only using a few free parameters (two or three). Also, the out-sample part of the test is reasonably long. Further more, we have not tested similar strategies before, so this strategy has not been picked from other similar ones that have failed; in other words, we have not cherry picked this strategy.

Even if the strategy has been built in a way that tries not to introduce bias, there could still be subtle flaws present.

The pattern we are using in the strategy has been found in the in-sample period, which is ten years long, but it has been observed mostly at the end of it.

For example, the trade we took at the beginning of the ten-year historical window (in the sample period) was using information that was not available at that time.

There is a different way to overcome this issue and it is an easier one. It is to discard those patterns that are not equally present along the time window.

Another way consists of using in the backtest information only after it becomes statistically significant.

For example, if we fit a parameter using data from 2000 to 2005, that parameter should not be used to trade historical data before 2005.

If you want to know more, we offer a comprehensive algo trading course. For more information, click here.