Sunday, November 4, 2012

The Future of Political Journalism

Nate Silver's predictions on FiveThirtyEight are starting to draw a lot of attention - and a lot of criticism. Almost all this criticism seems to be people interpreting statistical estimates as the blathering of a typical pundit, leading them  to very irrational claims, or concocting elaborate speculations why Silver cannot be right. They could not be more wrong, and in a few days I am not going to be shy about saying "I TOLD YOU SO".

(more after the break) 


Specifically, I want to say that there is something new coming. I think we are be seeing the beginning of a new scientific journalism, where specialists give statistically precise and mathematically unbiased estimates. It's pretty clear the election campaigns are moving this way already, micro-campaigning to the specific voters that might turn an election. The news media will need to follow suit, turning the political news cycle from a competition to entertain to a competition to accurately inform. 

That does not mean the pundits will go away. There needs to be representation of choices and strategy. Partisan opinion and ideological thoughts needs to be considered too. What needs to end are the outright lies, the disregard of the best available data, and opinions founded on no particular data at all.
A lot of people will really hate this, at least at first. We see this already in the Republican partisan criticism of Nate Silver. If Democrats had lost the last election and were predicted to lose this one, then I'm pretty sure it would be some liberals idiots accusing Silver of bias instead.

The cat is out of the bag though, and the value of Silver's methods are clear. Conservatives can't ignore it, they WON'T ignore it. By 2016 there will either be Conservative** statisticians offering their own analysis very similar to that of FiveThirtyEight, or the Conservative commentators (all of them, not just Conservatives) will come accept reality. The era where pundits can say anything and have no accountability for the accuracy of their claims is coming to an end.

** Statistician are already professionally conservative, meaning that statistical methods are adverse to error. Here I mean a politically Conservative statistician offering statistically conservative analysis.

7 comments:

  1. TA - right on. And you may want to check out the site that our oldest developed - the electoral map based on the InTrade online wagering data. It is similar to Silver's approach - aggregated data - but is the aggregate of tens of thousands of gamblers!

    http://electoralmap.net/2012/intrade.php

    the site is really really slow today - maybe traffic growing as the electin nears

    ReplyDelete
    Replies
    1. Hi Doc - Tell your oldest that is some nice work!

      I was just having a discussion about how Intrade and Los Vegas odds are likely to vary from statistical estimates. It's a different sort of polling, where the participant themselves weight their own data, as opposed to the statistical model. It's a very different approach, and I don't know to what extent it has been studied. It seems like it should be a combination of Game Theory and Statistics.

      Delete
    2. Will do. He commented that InTrade and Silver are both aggregators, with Silver being one guy and InTrade being thousands :-) He also said that he personally would have called Florida for Obama as did Silver, so InTrade was off by only one state.

      Delete
    3. TA - be sure to check out the update at electoralmap.net Douglas has added a summary graphic and comments - most interesting!!

      Delete
    4. Looks even better now!
      http://electoralmap.net/

      I am curious to see if the trading markets will now play a bigger role in other forecasting.

      Delete
  2. Yes, that should be interesting. Have you seen the Drunk Nate Silver meme that is making the rounds on the internet? Hilarious - I might post some of them.....

    ReplyDelete
    Replies
    1. I saw the earlier Tweet meme, but not this latest development. Good fun!
      http://theweek.com/article/index/236197/drunk-nate-silver-the-best-tweets-from-the-funniest-post-election-meme

      Delete