Can Pundits Actually Prognosticate? Answer: Mostly, No

from the nice to see some lookback dept

I’ve discussed in the past how loath the tech industry often is to ever look back at analyst “predictions.” The big research firms come out with all sorts of ridiculous predictions, and no one ever goes back and figures out how accurate they were. It seems that the same thing is frequently true with political prognosticators, so it’s interesting to see a Hamilton College public policy class analyze the predictions of 26 political pundits over the period of 15 months (September 2007 to December 2008) and measure how good the pundits were. It turns out most were “no better than a coin toss.” I’m not entirely convinced of the methodology, since it seems to make use of some subjective analysis from the description, but for the politically minded, it’s at least interesting to note that the most “accurate” pundits all fall on the left of the traditional political spectrum, while the least successful tended to fall on the right. I do wonder how much of that has to do with the timing (the period covered the financial decline and the Presidential election). It would be interesting to see a similar test run during a different period of time as well. I wonder if a similar analysis, say, prior to the election of a Republican president, would have turned up the opposite results. In other words: was there a fundamental quality in the predictions, or was it just that “the winning team” looks smarter in retrospect? Either way, it’s still great that people are going back and looking at how well some of these prognosticators did.

Filed Under: ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Can Pundits Actually Prognosticate? Answer: Mostly, No”

Subscribe: RSS Leave a comment
21 Comments
fogbugzdsays:

I always figured that if an “expert” was correct more than 50% of the time, then it was a win. It is easy for experts to get distracted by some detail and miss the big picture which is why it is much easier to be wrong than right. I watched Bill Gates pretty closely when he was still running Microsoft, and it looked to me that he was right a little over half the time. But that was enough to make him a billionaire.

Stevensays:

Unskilled and Unaware

I strongly suspect one of the major pitfalls of ‘professional journalism’ is exactly that they are trained in journalism. They then think this training grants them expertise in nearly all other areas of life. I would love to see a world in which journalism was only offered as a secondary major, and journalists reported on their primary fields.

There is a fascinating paper (pdf) here:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.64.2655&rep=rep1&type=pdf

It shows how people tend to vastly overestimate their abilities in an area they are unskilled. If you’ve never read it you should.

Of course seeing as how I’m unskilled in the field of journalism, it’s entirely possible I’ve fallen into my own trap.

ChurchHatesTuckersays:

Re: Unskilled and Unaware

I strongly suspect one of the major pitfalls of ‘professional journalism’ is exactly that they are trained in journalism. They then think this training grants them expertise in nearly all other areas of life.

This is so true it hurts.

We need to remember that almost all reporters are generalists, which means they’re relying on ‘sources’ to feed them information. Just about anything you read from mainstream sources should be treated as a press release. Because it usually is.

Ron Rezendessays:

For those who didn't know....

Loath or Loathe?

Loath is an adjective meaning “unwilling.” It ends with a hard th and rhymes with growth or both.

Loathe is a verb meaning “to hate intensely.” It ends with a soft th like the sound in smooth or breathe.

Examples: He was loath to admit that he was included in the deal.
(He was unwilling)

Alex loathes spiders.
(Hates them intensely)

Source: http://englishplus.com/grammar/00000238.htm
Copyright?1997-2006 English Plus, All rights reserved.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop ┬╗

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Loading...
Older Stuff
13:40 It's Great That Winnie The Pooh Is In The Public Domain; But He Should Have Been Free In 1982 (Or Earlier) (35)
12:06 Norton 360 Now Comes With Crypto Mining Capabilities And Sketchy Removal Process (28)
10:45 Chinese Government Dragnet Now Folding In American Social Media Platforms To Silence Dissent (14)
10:40 Daily Deal: The 2022 Ultimate Cybersecurity Analyst Preparation Bundle (0)
09:29 A Fight Between Facebook And The British Medical Journal Highlights The Difficulty Of Moderating 'Medical Misinformation' (9)
06:29 Court Ruling Paves The Way For Better, More Reliable Wi-Fi (4)
20:12 Eighth Circuit (Again) Says There's Nothing Wrong With Detaining Innocent Minors At Gunpoint (15)
15:48 China's Regulatory War On Its Gaming Industry Racks Up 14k Casualties (10)
13:31 Chinese Government Fines Local Car Dealerships For Surveilling While Not Being The Government (5)
12:08 Eric Clapton Pretends To Regret The Decision To Sue Random German Woman Who Listed A Bootleg Of One Of His CDs On Ebay (29)
10:44 ICE Is So Toxic That The DHS's Investigative Wing Is Asking To Be Completely Separated From It (29)
10:39 Daily Deal: The 2022 Complete Raspberry Pi And Arduino Developer Bundle (0)
09:31 Google Blocked An Article About Police From The Intercept... Because The Title Included A Phrase That Was Also A Movie Title (24)
06:22 Wireless Carriers Balk At FAA Demand For 5G Deployment Delays Amid Shaky Safety Concerns (16)
19:53 Tenth Circuit Denies Qualified Immunity To Social Worker Who Fabricated A Mother's Confession Of Child Abuse (35)
15:39 Sci-Hub's Creator Thinks Academic Publishers, Not Her Site, Are The Real Threat To Science, And Says: 'Any Law Against Knowledge Is Fundamentally Unjust' (34)
13:32 Federal Court Tells Proud Boys Defendants That Raiding The Capitol Building Isn't Covered By The First Amendment (25)
12:14 US Courts Realizing They Have A Judge Alan Albright Sized Problem In Waco (17)
10:44 Boston Police Department Used Forfeiture Funds To Hide Purchase Of Surveillance Tech From City Reps (16)
10:39 Daily Deal: The Ultimate Microsoft Excel Training Bundle (0)
09:20 NY Senator Proposes Ridiculously Unconstitutional Social Media Law That Is The Mirror Opposite Of Equally Unconstitutional Laws In Florida & Texas (25)
06:12 Telecom Monopolies Are Exploiting Crappy U.S. Broadband Maps To Block Community Broadband Grant Requests (7)
12:00 Funniest/Most Insightful Comments Of 2021 At Techdirt (17)
10:00 Gaming Like It's 1926: Join The Fourth Annual Public Domain Game Jam (6)
09:00 New Year's Message: The Arc Of The Moral Universe Is A Twisty Path (33)
19:39 DHS, ICE Begin Body Camera Pilot Program With Surprisingly Good Policies In Place (7)
15:29 Remembering Techdirt Contributors Sherwin And Elliot (1)
13:32 DC Metro PD's Powerful Review Panel Keeps Giving Bad Cops Their Jobs Back (6)
12:11 Missouri Governor Still Expects Journalists To Be Prosecuted For Showing How His Admin Leaked Teacher Social Security Numbers (39)
10:48 Oversight Board Overturning Instagram Takedown Of Ayahuasca Post Demonstrates The Impossibility Of Content Moderation (10)
More arrow
This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it