Suddenly, Statistics in the Recall is in the News

I don't know that I can take credit for this, but this morning Phil Scarr at Blogging Blue points me to an article at the Milwaukee Journal Sentinel:

Analysis: Invalid signatures likely not enough to halt Walker recall

This seems to be just what I suggested in my previous post, two days ago. Whether or not I had the idea first, I applaud the effort.

[Edit: Fixed a link. I originally linked to a different but related post by Phil.]

The Statistics of Verifying Recall Signatures

There is a huge political battle raging to Wisconsin, but you probably know this already. The drive to recall Governor Scott Walker has gotten plenty of media attention. Some 540,000 signatures are needed, and the Democrats turned in well over one million signatures to be verified. Walker supporters are hard at work trying to identify false signatures, to get as many petition thrown out as they can. That means about half of all signatures could be bad and the Democrats would still have enough to force a recall election. Given this rather daunting situation, how hard should Walker's supporters try? Do they really need to check 500,000 signature, a million signatures? How many is enough to be confident of a reliable result?

Regardless of your opinion about Scott Walker and the recall, some simple statistical sampling can help answer the question, and it requires checking far fewer than 500,000 signatures. I'm going to assume that each petition form contains 20 signatures, and 50,000 petition forms to total one million signatures. Essential to this process is a simple random sample**, where we can select a sample of petitions so that each form has an equal chance of being selected. There are fancier schemes, but this is the easiest way get an unbiased sample, and for me to explain.

Starting with a generous assumption that half of all signatures are fake, and that this number varies with a standard deviation of 2.6 bad signatures per petition, meaning most petitions have between 5 and 15 bad signatures (also generous). For a sample of size n petitions that gives a total count of x bad signatures the formula for the percentage bad is x/(20 * n) [x divided by (20 times n)] with a standard error of 2.6/sqrt(n)  [that's 2.6 divided by the square-root of n]. Based on the mathematical law of statistical averages from a random sample, we can say that the actual number of bad signatures is close to x/(20 * n), where close means it is within about 2 standard errors in 95% of all such samples performed in this way. For a sample of size n = 1000 petitions, our assumptions and statistics say we should observe 50% plus or minus about 3.2%, or between 46.8% and 53.2% bad signatures - IF the assumptions are correct. We can say there is a confidence level of about 95% (19 times out of 20) that the interval generated this way will capture the true rate of false signatures.


Now the good news: If the actual percentage is more or less than 50%, the standard error should be a little smaller either way, meaning the estimate will be a little more accurate. More good news: this setting is what statisticians call a "finite population sample", which means this estimate will be a little more accurate yet, because the recount is sampling a significant fraction of the total population.


Long story short, if you want to verify recall petitions, take a sample of about 1000 petitions, check them carefully, and calculate the percentage of bad signatures. If that percentage is less than 45% or so, then it is time to stop counting and start campaigning.


** In practice random samples are not always "simple", but this is what statisticians call it.

Statistics for Badgers

I just discovered BadgerStats,  an organization that presents data driven commentary on Wisconsin's economy, education, business climate, and other topics.

http://badgerstat.org/2011/jobs/

Our work is motivated by our belief that:

• Many Wisconsinites want credible, nonpartisan information about their state, including insights into what’s working and what’s not, and about our state’s challenges and opportunities.

• Meeting Wisconsin’s challenges will require government that is more efficient andeffective, producing better results for citizens and better value for taxpayers. 

• Wisconsin government, at the state and local levels, would benefit from a more performance-oriented culture that focuses on results and uses performance data to manage.

• Every citizen deserves to know how their government is doing in key policy areas. Toward that end, every level of government (and every agency) should provide, online for citizens, a set of clear, timely, and accurate performance measures and goals. 

• Wisconsin’s future depends on an informed citizenry, since meeting our state’s challenges — and seizing our opportunities — will require people of all political stripes to come together in informed public dialogue to help chart our future.

Speaking as a data-guy, I appreciate and encourage this sort of information oriented reporting. This could become Wisconsin's own version of 538.com.

http://badgerstat.org/

Drinking Liberally in Waukesha

I just returned from the local Drinking Liberally event, which I learned about from reading local political blog Waukesha Wonk. Lately I've been to buried in work to be very motivated about anything, much less politics. However, this meeting was very informative about some behind the scenes goings-on, and it makes me realize how little I know about how politics actually works. The more I know, the more I want to make it work differently.

The guest speaker was Scott Wittkopf (Badger Democracy), who lead the discussion, shared some of his insights, and tipped us to some recent events. Scott is soft spoken and well-informed. I haven't been reading his blog previously, but it was clear he put a great deal of time into research for his blog and the articles he writes. If I had to pick one thing I came away with from this session, it new sense for the corporate political machine that has its grip on Wisconsin (for that matter, the nation), and the vital need to fight back against it.

Despite the "liberal" in Drinking Liberally, the event was non-partisan. People there seemed reasonable, and all were concerned about how wrong things have gone politically in Wisconsin. My thanks to Lisa and Phil for organizing, and the Spizzo Cafe for hosting. I plan to drink liberally* again next month.

* full disclosure - two beers is pretty liberal drinking for me nowadays.

Science Denial, at Darryl Cunningham Investigates

Simply too good not to share.

Image: Darryl Cunningham Investigates

That's merely the first in a series. Take a few minutes and read the whole thing.

Science Blogs Survey

I just participated in the Science Bloggers Census. There was some question in my mind if this blog really qualifies, but Edward was kind enough to link me to an article answering that question of "Just what is a Science Blog?".

Looking back at my recent posts, I haven't been posting very much science related content recently, but I'm also counting some of my mathematical posts at my other blog. That blog gets more traffic, and in a way it's even more scientific than what I write here, but most people don't consider games as science.

The survey results will appear in September at http://labs.fieldofscience.com/ .

Crisis in Dairyland

Some commentary on our local politics from the Daily Show.

Crisis in Dairyland - Recall Edition