Sunday, November 4, 2012

The Future of Political Journalism

Nate Silver's predictions on FiveThirtyEight are starting to draw a lot of attention - and a lot of criticism. Almost all this criticism seems to be people interpreting statistical estimates as the blathering of a typical pundit, leading them  to very irrational claims, or concocting elaborate speculations why Silver cannot be right. They could not be more wrong, and in a few days I am not going to be shy about saying "I TOLD YOU SO".

(more after the break) 

Thursday, November 1, 2012

More Survey Analysis

I'm repeating my previous post, since it seems relevant to review the data again before the election. I accessed all these sites at about 9:00 PM CST. I

Real Clear Politics
Obama 201
Romney 191
Toss Ups 146
Obama by +1 ECV

Intrade Presidential Election 2012
Obama ~67% win
Romney~33% win

FiveThirthyEight 
Obama 303.2 +/-56 ECV
Romney 234.8 -/+56 ECV
~81% Obama wins

HuffPost Pollster

Obama  277 ECV
Romney  206 ECV
Toss Ups 55 ECV
Obama has enough certain electoral votes to win.

Obama 285 ECV
Romney 191 ECV
Toss Ups 62 ECV (but 44 of those lean strongly towards Romney)

Election Analytics 
Obama 296.7
Romney 241.3
~99.4% Obama wins

As before, I have arranged these in roughly increasing order of favor for Obama winning the 2012 election. The last three sites (HuffPost, TPM, EA) are making strong claims that Obama has the electoral votes to win already. 538 is not far behind that claim.
RCP seems to be sitting on the fence, not making strong claim about the toss-up States, and there is nothing wrong with that.

Other notes:
HuffPost Pollster has been added to the results. (but you saw that already.)

FiveThirtyEight: Last time I made the claim that Nate Silver's analysis is as close to neutral as can be found. Today though, I saw this: Nate Silver bets Joe Scarborough $1000 that Obama wins. It is not clear if this was intended as a partisan statement, or simply a good bet. It's not wrong to claim Obama is a good bet.

Should I update again tomorrow?

Tuesday, October 30, 2012

Survey Says ...

After discussing politics with a friend this morning, I was inspired to make the rounds of sites that accumulate political polling results and see what they all are saying. All results accessed on the web late afternoon on November 6th.

Real Clear Politics
Obama 201 +/-61
Romney 191 -/+61
Toss Ups 146
Obama by +10 ECV
(edit: those error bound probably belong to the 538 analysis, not RCP).

Intrade Presidential Election 2012
Obama 64.6% win
Romney 35.4% win

FiveThirthyEight 
Obama 294.6 ECV
Romney 243.3 ECV
~72.9% Obama wins

Obama 265
Romney 206
(Comment: therefore 67 are not certain)

Election Analytics 
Obama 291.4
Romney 246.6
~95% Obama wins

I have arranged these in increasing order of favor for Obama winning the 2012 election. Real Clear Politics and Talking Points Memo are sites I do not regularly follow, so I have no clear opinion of their methods. I suspect these sites have a conservative and liberal biases (respectively) but I have not direct evidence for that. Certainly if they are both presenting statistical results in an unbiased manner, they ought to have about the same conclusion. All I can really say here is that RCP doesn't offer much of a direct election forecast.

Edit: Intrade is another site I don't follow, but it has been getting a lot of media attention too.

I personally favor the methods Nate Silver uses on FiveThirtyEight, as I consider it to be the most technically sophisticated analysis, and it takes economic data into account as well. Silver offers a lot of day-to-day commentary on daily polling and his predictive model. Since the Democrats have lead the elections since the blog started in 2008, many of those prediction have been that Democrats are going to win. Some people interpret this an a liberal bias, but Silver's Senate predictions have been very good, and it not a bias if he is correct.  I have never seen Silver make anything close to an endorsement, and so on that basis I think FiveThirtyEight gives the most unbiased political analysis available.

Election Analytics is different from the others. It's a small academic page rather than a news site. Their statistical methods are sound, but they make some strong assumptions, which seems to be why they are able to make such a strong prediction for Obama winning. I haven't looked into just what these assumptions are, so I can't say if they are justified or not.

All this might not make my friend happy, but all the data is saying pretty much the same thing, with varying degrees on certainty.

Update: Added Intrade prediction at the suggestion of +Kevin Clift. Accessed the morning of 10/31/12.
Edit: next time I'll include this site too: http://electoral-vote.com/

Saturday, August 4, 2012

The Creationist 419 Scam

You would think that outrageous claims are so likely to be rejected that the person making the claim would just give up and go away. For an example of this you might check out this Sensuous Curmudgeon post "ICR: Plants Are Not Alive". The Institute for Creation Research claims that because plants do not move and do not have blood, they are not alive, and they justify this based on the Old Testament and a some quadruple backwards spinning logical somersaults that would make Gabby Douglas gawk. There are plenty of other examples, but I won't belabor the point. As my buddies at The Sensuous Curmudgeon often note, the scammers* have to know they have no scientific standing, but they do it anyway. WHY?

Consider a known scam that everyone can agree is a scam; one that is no farther away than your email SPAM folder. Microsoft scientist Cormac Herley has a paper out:


Edit: Original link seems broken. Try this instead.

... dissecting the mathematics of the Nigerian 419 scam. The Wall Street Journal Online has a less technical summary, see "Why We Should Scam the Scammers".

Here is a brief quote from Herley, with my emphasis added:

"... Far-fetched tales of West African riches strike most as comical. Our analysis suggests that is an advantage to the attacker, not a disadvantage. Since his attack has a low density of victims the Nigerian scammer has an over-riding need to reduce false positives. By sending an email that repels all but the most gullible the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor."

 Here is Herley again, later on:

"Since gullibility is unobservable, the best strategy is to get those who possess this quality to self-identify. An email with tales of fabulous amounts of money and West African corruption will strike all but the most gullible as bizarre. It will be recognized and ignored by anyone who has been using the Internet long enough to have seen it several times.  [ ... ]  It won’t be pursued by anyone who consults sensible family or fiends, or who reads any of the advice banks and money transfer agencies make available. Those who remain are the scammers ideal targets."
It's brilliant actually. Finding people susceptible to a scam is hard, but weeding out those least susceptible is as easy as concocting a lame story. The more outrageous the tale, the less likely it is to attract those who can see through it, leaving those who are mostly likely to be successfully fleeced by the scammer.

There's is a shorter summary, and an older one; Abraham Lincoln put it like this, "You can fool some of the people all of the time, and all of the people some of the time, but you can not fool all of the people all of the time."



The scammers know that most people will apply some amount of logic and reason and reject the obviously incorrect, and this is what they want. They want to weed out the majority who will never buy into the scam, and speak to the few they might fool. When the scammer is the ICR and the marks falls for the false dichotomy that religious belief must overrule scientific knowledge, the scam is particularly insidious.

Herley suggests a response to the 419 scams, to counter-SPAM the scammers with automated responses, false positives that waste time and money and take the profit out of the scam. This would be harder to apply to Creationist scammers, requiring a large number of people (or automated facsimiles) to "Go Poe" and troll the Creationists where they live. That doesn't sound like fun, and it doesn't strike me as ethical. Still, the suggestion has been made before.

* I'd like to make distinction between those who hold to Creationist belief and those those making claims in support of Creation science. The former may hold a sincere belief, but the latter are deliberately lying in an attempt to undermine science and science education.

Sunday, July 1, 2012

Wrong and Wronger

A month ago I wrote about telling off a Nobel laureate (see Wrong). That is a conceit on my part, because I doubt Dr. Josephson will ever read it. Someone read it though, because it sparked a flurry of response from some Answers in Genesis Creationists (yes, I know that is redundant). One in particular, identifying himself only as Bonesiii Dromer, was moved to a truly spectacular display of sanctimonious bombast. After some thought and considerable delay (I was exceptionally busy) I finally posted a response to the nonsense.

--- More after the break ---

Blog changes

I just made a complete overhaul of my blog template, because I found that quoted text just wasn't at all legible, and other formatting annoyances. No doubt this will cause chaos and destruction throughout the blogosphere. Please let me know if you spot anything in need to fixing.

Friday, June 29, 2012

Tomatoes!

How can I not link to this Discovery Blog Post?


Enjoy.














Hey. Where did she come from?