A Statistically Significant Edge

Last week we discussed that just because a trading system is making money doesn't prove it has an edge.  It means you have both the chance that the system has an edge - and also that it may just be lucky.

One of our advisors wrote back to see if they understood that approach.

 

The odds of flipping a coin and getting heads 25 times in a row is roughly 1-in-33 million. So if we have 33 million flippers and 100 get 25 heads in a row, statistically that is very improbable.  We can deduce that group of 100 is a combination of some lucky flippers, but also that some have a "flipping edge."  We may not be able to say which is which, but as a group our 100 will still consistently provide an edge in future flip-offs.

 

Well, that is correct.  In fact, if we were developing coin-flipping agents that would be as far as we would be able to go.  However, we are in luck because our problem has an extra dimension, which makes it possible to filter-out some of the "lucky" Bots from our trading systems.

 

Determining Which are the Best Systems.

 

There are several ways to determine whether a trading system has a persistent edge.  For example, we can look at the market returns during the trading period and compare and contrast that with our trading results.

This is significant because many systems have either a long or short bias.  That means even if a system does not have an edge, it would be more likely to turn a profit when its bias is in alignment with the market. 

We try to correct that bias using some math and statistical magic, in order to determine whether the system has a predictive edge.

 

It Is a Lot Simpler Than It Sounds.

 
Imagine a system that picks trades based on a roulette spin.  Instead of numbers or colors, the wheel is filled with "Go Long" and "Go Short" selections.  As long as the choices are balanced, the system is random.  But what if the roulette wheel had more opportunities for "long" selections than "short" selections?

This random system would appear to be "in-phase" whenever the market is in an uptrend.  But does it have an edge? 

One Way To Calculate Whether You Have An Edge.
 

Let's say that you test a particular trading system on hourly bars of the S&P 500 Index from January 2000 until today. 

 

1. The first thing you need is the total net profit of the system for all its trades.  
2. The second thing you need to calculate is the percent of time it spent long and short during the test period.
3. Third, you need to generate a reasonably large population of completely random entries and exits with the same percentage of long/short time as your back-tested results (this step can be done many times to create a range of results). 
4. Fourth, use statistical inference to calculate the average profit of these purely random entry tests for that same test period. 
5. Finally, subtract that amount from the total back-tested net profit from the first step.

 

According to the law of large numbers, in the case of the "roulette" system illustrated above, correcting for bias this way, the P&L of random systems would end up close to zero … while systems with real predictive power would be left with significant residual profits after the bias correction.

While, the math isn't difficult … the process is still challenging because it takes significant resources to crunch that many numbers for hundreds of thousands of Bots. 

The good thing about RAM, CPU cycles and disk space is that they keep getting cheaper and more powerful.