Models are representations of the real world. They help you better understand the real world by breaking it into pieces, making them good when you need to assess a risk or make a decision. But as Kevin Madigan points out, in an article on Property Casualty 360, a National Underwiriter’s website, no model can cater for every contingency and some models are better than others at helping us assess information about a risk or decision. The main thing is not to use models unquestioningly, for two reasons:
- Models are based on underlying assumptions.
- Models work on probabilities.
That means you need to understand both the assumptions models make and how they calculate probabilities.
First, ask yourself what your model’s underlying assumptions are and how correct and relevant are they to your organisation or decision. What contingencies are built into and left out of the model? Are the missing pieces important and if so, how can you incorporate them into your decision-making or risk management?
Next, find out how the model calculates probabilities. There are two ways: ‘classical’ probabilities and ‘subjective’ probabilities. You can be reasonably confident in classical probabilities because they are based on observation and experimentation; for example, flipping a coin or testing a drug on a target group and a control group. (Why not call them objective probabilities? Good question; too logical maybe.)
But you can’t experiment or observe elements of decisions about unusual events or problems or of catastrophic risks. That’s when subjective probabilities are used. Either you or the model need to estimate probabilities, perhaps based on observations about the past, informed assumptions about the future, and your ‘best guess’. That’s a long way away from classical probabilities.
So use models to help you make decisions and calculate risks but use them all with care, a questioning mind, and common sense:
- Don’t take any model at face value.
- Don’t interpret any model, especially those using subjective probabilities, as factual.
The statistician George EO Box put it well:
‘All models are wrong but some are useful.’
P.S. When you’re working with models, here’s a phrase guaranteed to impress: Don’t get caught up in delusional exactitude. In other words, be wary of models that claim to have a high degree of precision.
What models do you use in your work? How accurately do they break information into pieces and represent the real world? What are their underlying assumptions and how relevant are they to the situation you’re applying them to? What type of probabilities do they use? In what ways might the models you use be wrong?