There appears to be two binary opinions with regards to the overall value of backtesting and that opinion is evenly divided. Many traders swear by backtesting, others cite it as patently useless, claiming that only forward testing in real time in the market place works. But is there a critical logical fault in the claim that only forward testing works? After all, there’s no guarantees that forward testing will produce positive results and can cost both time and money, unless we’re using a demo account.
We may have hit a sweet spot using our short term, forward (live) testing method to work, but then watch it crash and burn shortly thereafter and if we’re a swing trader, how long can we wait in order to determine that our forward testing has worked? Three months? That’s potentially a lot of profit we could be missing out on. Whereas our backtesting may have looked promising, we’ve then utilized it in the market, having the opportunity to benefit from the market during that same three month period.
History will teach us nothing; presenting the case against backtesting.
Those who claim that backtesting doesn’t work will cite that traders will simply attempt to curve fit their strategy to accommodate potential flattering results. The accusation is that they won’t have the self-discipline, even using historical data, to let the data run freely without interference, to produce the pure results. Critics claim that if traders witness the backtestto fail in delivering the required results, then the temptation to curve fit is overwhelming; the trader will micro adjust the various parameters and indicator settings in order to get the required result, to then run with the strategy in live conditions. Particularly if the strategy relies on a series of indicators to align, to deliver the signal to enter or exit the marketplace, the temptation to fine tune can be heightened. For example; back testers will alter the settings on an indicator, or alter the timeframe, or find a security that fits the strategy, if they’re convinced of its validity, rather than simply abandoning the method/strategy and moving on.
Presenting the case for backtesting.
“History doesn’t repeat, but it does rhyme” is a phrase many of us are familiar with, in relation to trading. If that phrase has some value, in relation to trading patterns rhyming, then surely backtesting has value? Whilst we accept that no two moments in the markets are the same and that each security’s movement (however short or long) is unique, surely if certain technical conditions chime on a certain time frame, then there’s a reasonable prospect for those conditions and the subjective price behaviour of a particular currency pair, to take on a similar pattern in the future?
We have no guarantee that forward testing will work, as we’re faced with blank space on the right of our chart, but we do have some historical evidence through backtesting, on which to base our judgment. Therefore with forward testing it could be argued that we’re basing our test on future events, but with backtesting (to then use the backtest results live in the market), we’re using historical evidence, which surely has relevance and value, over and above only future expectations.
Using backtesting and forward testing for optimum results.
If we were lab technicians, testing out theories before reaching a conclusion, would we firstly analyse previous results and historical precedents, before planning further highly detailed forensic tests? Perhaps here’s our solution; we use a combination of back and forward testing, in order to test our theories and reach our conclusion.
Let’s imagine a scenario whereby our backtest has proven successful, and we can establish the results of a backtest using the various proprietary tools available to us fairly quickly. Our next challenge should be how long to gauge the efficiency of our results live in the markets. As previously mentioned; if the forward test is too long, then we could be wasting time and potentially missing out on unclaimed profit. Therefore we need to determine the right time scale by which to judge our forward test.
Naturally, if we’re a day trader, then we’ll take more trades during our back and forward testing, than if we’re a swing trader. If we backtest over a three month period we can’t possibly forward test using our normal trading parameters, over such a period of time. With day trading we may take 100 trades during a month, with swing trading it may be lower than 5 and that’s assuming (for both tests), we’re trading two major currency pairs. Can we realistically make a judgment based on 5 trades? Moreover, can we forward test our theories based on 5 trades? Unlikely and what’s more; even waiting a month for forward testing in live mode to deliver results, can prove to be too long.
Therefore, we also have to recognize and realize the limitations of our backtest in relation to our trading style. Perhaps we also have to recognize that discovering the balance between backtesting and forward testing, is only suitable for certain trading styles. Arguably it’s scalping or day trading, were the likelihood of rhyming is most likely to be applicable.
