Making Sense of Market Models

We use models every day to make decisions—from simple ones like “nachos taste good” to complex ones like quantum mechanics. Models help us understand and navigate the world. However, no model is perfect. They’re simplified representations of reality and come with limitations. Overconfidence in a model can lead to mistakes, so it’s crucial to know both its strengths and where it breaks down.

Take financial markets, for example. They are vast and ever-changing, making it impossible to engage with them without using a model. The key is to use the right models and interpret their results thoughtfully. This simplifies complex systems and helps you make informed decisions.

Tradeoffs in Using Models

One challenge with models is the risk of incorrect inputs—either due to user error or misunderstanding the model. This is often referred to as “garbage in, garbage out.”

For instance, consider Modern Portfolio Theory (MPT). This framework emphasizes the overall risk and return characteristics of a portfolio rather than focusing on individual assets. It promotes diversification and has significant implications for building portfolios. However, it’s easy to misuse. Some parts of the financial industry exploit MPT, and when things go wrong, they blame the model. But if you misuse a model, it’s not the model’s fault.

To evaluate a model, ask two key questions:

• Is it testable? Can you check the predictions it makes?
• Is it useful? Does it provide meaningful insights?

If the answer to either question is “no,” the model is likely not worth using. Even for useful models, remember that no model is perfect. Newtonian physics, for example, works well at everyday scales but breaks down with very large or very small objects. It’s still taught in schools because it’s practical for most situations. Choosing the right model depends on understanding your question and finding a model suited to answer it.

Type I vs. Type II Errors

Even the best models involve tradeoffs. When making decisions, there’s almost always uncertainty, and mistakes are inevitable. These mistakes can take two forms:

1. Type I error (false positive): Saying “yes” when you should have said “no.”
2. Type II error (false negative): Saying “no” when you should have said “yes.”

For example, when the FDA evaluates a new drug, it faces these tradeoffs. Approving a harmful drug is a Type I error while rejecting a beneficial drug is a Type II error. Balancing these risks depends on the context. A Type II error might be more costly for life-saving drugs than a Type I error.

In investments, we face similar decisions. When evaluating whether to add a new asset to a portfolio:

• A Type I error would mean adding an asset that doesn’t provide the expected benefit.
• A Type II error would mean missing out on an asset that could improve the portfolio.

Studies suggest prioritizing minimizing Type I errors (avoiding bad ideas), as good investment opportunities are rare. Type II errors (missing good ideas) are less costly than Type I errors (implementing bad ideas).

Avoiding Data Mining and False Insights

One common source of Type I errors is data mining. Financial markets are noisy, so it’s easy to mistake random patterns for meaningful trends. Therefore, it’s not enough for a model to produce results. We need to understand why those results make sense. Without a solid risk-based explanation, we can’t trust the results.

Every decision requires weighing potential benefits against risks. Even promising new strategies should be evaluated cautiously to avoid unnecessary portfolio changes. Good ideas are rare and often provide diminishing returns, so we set a high bar for what gets included.

Confidence in your investment process is crucial for long-term success. Markets will rise and fall, and media noise can tempt you to make impulsive changes. But with a solid model guiding you, you can stay focused and weather market fluctuations. By understanding the strengths and limitations of models, you can make informed decisions, avoid costly mistakes, and stay on track toward your financial goals.