Part 1

Common Errors in Statistical Analysis

Errors is common. Check that: Errors are common

It’s a commonality among everyone: We all make mistakes, whether it is something as minor as mistyping your login password or misplacing your phone. Everyone makes errors…especially in the ruthless game of investing. 

In his memo to clients “It’s All a Big Mistake,” Howard Marks writes that “mistakes are all that superior investing is about.” To grow and prosper, you have to make errors and fail. When you get knocked down, face first on the ground, you push yourself up and brush the dirt off. The concept applies to investing, where losses, bad luck, and bad timing are inevitable for even the best of the best. 

Common Errors in Statistical Analysis

While the memo is directed toward investing error, the principle applies for statistical analysis as well. Statistical analysis is the collection and interpretation of data in order to uncover patterns and trends. Something as simple as calculating the mean percentage grade of a high school math class falls under the tab of statistical analysis. 

A statistical error is the unknown difference between the obtained value and the actual value. In his memo, Marks writes, “the potential of error is ever-present, and thus of the importance of mistake minimization as a key goal.” When performing analysis, there is a guarantee that errors will occur at some point. However, recognizing where the mistake occurred and avoiding the duplication of that error in future analysis remains the ultimate goal. 

You might think the tests used for statistical analysis provide 100% accuracy. But that’s not the case. To perform an analysis, consider factors such as sample size, external bias, sampling methods, etc. Each is needed to achieve the most accurate data. Bottom line: Many errors can occur in any analysis. It is critical not to fall into the same trap. Going through each tedious step of the analysis process is difficult to conduct with precision. But without it, the flood gates are left open for the rapid currents of errors. 

In June, we polled our research heads as the market had fallen once again amid the 2022 first-half bear market. We asked each of them: How do you manage emotions amid downturns? Brian Rauscher, our Head of Global Portfolio Strategy and Asset Allocation, and Adam Gould, Head of Quantitative Research, both offered timeless insight on mistakes — and the inevitable nature of making errors. 

Said Rauscher: “We hear, ‘When the market is exuberant, we want to be careful, and vice versa.’ Sometimes, you must throw analysis out the window. Tops and bottoms are made by fear and greed. At every major top, markets have been overvalued. At every major bottom, markets have been undervalued. Long-term, I believe markets are driven by corporate earnings. Bottom line: you should have a system and stick with it to be disciplined. Using tools to remove not all but good portions of emotion is important, though having some degree of subjectivity is important. Also, wait for your opportunities. As I’ve grown older, I’ve learned how patience helps. Markets overshoot to both sides, which creates challenges, but opportunities as well. Proper sizing and rebalancing are critical actions. Another thing: there’s almost always going to be a bull market in something. People have regrets, ‘Oh, I shouldn’t have missed out on that.’ Guess what? The market will always be here. Miss on something? Big deal! Show up and look for the next one.”

Added Gould: “Trust your plan, trust your models. As good as your plan might be, sometimes you’re going to underperform. You want to have a plan and stick to it. Know your limits and principles. My model, for example, had a great couple of months, and it struggled a little in June. I’m not going to suddenly change it. It happens. Do you have the courage to stay the course? Say you’re invested in Bitcoin. How do you look at that and not get upset? You can’t. It’s about staying the course.”

In his famous quote, Thomas Palmer summed it up perfectly:“ If at first you don’t succeed, try, try, try again.” 

Related Guides