A few days ago I blogged about Newsweek‘s lame-acre retraction of a story they published 20 years ago. The story claimed unmarried women over 40 were more likely to die by a terrorist attack than find a husband. The claim was bogus; Newsweek‘s attitude contemptible.

Now comes this article by Philip E. Tetlock at Project Syndicate,  “How Accurate Are Your Pet Pundits?”

Tetlock has written a book (Expert Political Judgment: How Good Is It? How Can We Know?) that looks at pundit predictions, reported in the media, and whether they’ve come to pass.

This guy deserves a medal for calling attention to this. A big flashy platinum medal with a huge cash prize attached:

[W]hen hordes of pundits are jostling for the limelight, many are tempted to claim that they know more than they do. Boom and doom pundits are the most reliable over-claimers.

Between 1985 and 2005, boomsters made 10-year forecasts that exaggerated the chances of big positive changes in both financial markets (e.g., a Dow Jones Industrial Average of 36,000) and world politics (e.g., tranquility in the Middle East and dynamic growth in sub-Saharan Africa). They assigned probabilities of 65% to rosy scenarios that materialized only 15% of the time.

In the same period, doomsters performed even more poorly, exaggerating the chances of negative changes in all the same places where boomsters accentuated the positive, plus several more (I still await the impending disintegration of Canada, Nigeria, India, Indonesia, South Africa, Belgium, and Sudan). They assigned probabilities of 70% to bleak scenarios that materialized only 12% of the time.

Tetlock calls for tracking pundits’ records publicly, so that media consumers have a way of judging their credibility.

I say, we should also hold the media to account. After all, if false pundits weren’t hugged and kissed and led out into the spotlight, “here’s your microphone, dear,” their silly pronouncements could do no harm . . .