I surveyed my co-workers with this question. Keep in mind that my co-workers have a higher knowledge of statistics than the general population. I got 33 responses, 24 of which were Coin B (73 percent). If you agree, then you’re in agreement with some smart people. Unfortunately, you’re also wrong. Don’t worry, I would have been wrong too, and I love stats. I should mention that several people that got the answer right admitted that they just assumed it was a trick question.
Most people think that coin B is more likely to be fair because it came up heads at a rate closer to 50 percent. Coin A came up heads “only” 30 percent of the time and is therefore more likely to be unfair.
The flaw in this logic is that it ignores the law of large numbers. It places too much emphasis on the small sample, and too little emphasis on the large sample. When assessing outcomes, people are susceptible to many biases. I want to focus here on the tendency to pay too much credence to short-run results. If you flip a fair coin 10 times, there is actually a 17 percent chance that it comes up heads 3 times or less. Basically, any result is reasonable over the course of 10 coin flips. However, if you flip a fair coin 1,000 times, there is less than a 0.1 percent chance that it comes up heads 450 times or less. After 1,000 flips, we can be fairly confident that coin B has a true expected outcome somewhere between 42 and 48 percent heads, and therefore is not fair. In polling, this would be called a margin of error of 3 percent
As the number of coin flips increases, there is much smaller chance of outlying events. Coin B is therefore much more likely to be unfair. We have such limited data for coin A that we cannot say whether it’s fair or not.
How is this relevant? Investors (and Morningstar, and Barron’s, and Money Magazine, and…) give credence to an investment manager’s three-year or five-year track record. Naturally, certain managers will outperform over these short-run periods, even if just by chance. However, these trends should not be extrapolated to a long-run trend.
There has been considerable research that the number of mutual funds that outperform their benchmarks is actually the same or worse than chance would predict. One such study looked at 2,862 broad, actively managed domestic mutual funds that were operating for at least 12 months through 2010. The study looked at the funds that were in the top quarter of performance for five straight calendar years.
Let’s look at what would happen by chance alone. At the end of 2010, there were 715 funds with performance in the top quarter (25 percent of 2,862). By chance alone, if 25 percent of those remained in the top quarter, there would be 179 funds that were in the top quarter for 2010 and 2011. There would remain 45 funds in the top quarter for three years, and 11 funds in the top quarter for four straight years. Finally, by chance alone, we would expect somewhere between two and three funds to be in the top quarter of funds for five straight years.
(1/4) x (1/4) x (1/4) x (1/4) x (1/4) x 2862 = 2.79
How many funds in the study actually had performance (relative to a benchmark) in the top quarter for five straight years? Exactly two, precisely what you’d expect by chance alone. How are those funds doing this year? Through mid-November, the SouthernSun Small Cap Fund (SSSFX) was down 12.7 percent versus 4.1 percent for its benchmark Russell 2000 Index. The Hodges Small Cap Fund (HDPSX) had performance nearly identical to its benchmark. It is likely that this study of 2,862 mutual funds will have zero funds in the top quarter of relative performance for six straight years.
Examples of this bias are everywhere. I got the flu shot last year and I still got the flu. Does that mean that I shouldn’t get the flu shot this year because it doesn’t work? Of course not. I can’t extrapolate the probability of getting the flu from a single year, although lots of people do. How about the people with a grandparent that smoked a pack of cigarettes a day and lived until age 90? Cigarettes can’t be that bad based on that sample of Grandpa Bob.
Investing for retirement should be a long-term process. It would be nice to extrapolate meaning from three- or five-year performance, but more often than not we may actually be chasing returns that are likely to regress to the mean in the future. As always, the lesson is that you shouldn’t pay for past performance.
Click here to sign up for the Daily Economy weekly digest!