Predicting the future

How much reliance can we place on regression to the mean in judging what the future will bring? What are we to make of a concept that has great power under some conditions but leads to disaster under others? Keynes admitted that “as living and moving beings, we are forced to act … (even when) our existing knowledge does not provide a sufficient basis for a calculated mathematical expectation.”

With rules of thumb, experience, instinct and conventions – in other words, gut - we manage to stumble from the present into the future … The trick is to be flexible enough to recognize that regression to the mean is only a tool; it is not a religion with immutable dogma and ceremonies. Used to make mechanical extrapolations of the past … regression to the mean is little more than mumbo-jumbo. Never depend upon it to come into play without constantly questioning the relevance of the assumptions that support the procedure. Francis Galton spoke wisely when he urged us to “revel in more comprehensive views” that the average.

Peter Bernstein, Against the Gods