jackman.stanford.edu/blog
• Bayesian Analysis for the Social Sciences Wiley; Amazon; errata as of 5/23/13

• 113th U.S. Senate
• ideal point estimates pdf csv 7/23/14
• scatterplot against 2012 Obama vote share pdf
• roll call object: RData
• 113th U.S. House
• ideal point estimates pdf csv 7/24/14
• scatterplot vs Obama vote share pdf d3/svg
• roll call object: RData

## Friday September 6, 2013

Filed under: Australian Politics,statistics — jackman @ 2:18 pm

I’m looking to see if I have missed something, but I think thats it. I’ve got the releases yesterday, including the 54-46 results from Newspoll and Nielsen:

Entering all this into the poll averaging model produces an ALP TPP estimate of 46.2%, +/-0.9. The 90-day trajectory (click for larger view):

This estimate is formed as a 67/33 combination of

1. the model that is constrained to fit the 2010 election result exactly, and produces a set of house effects that are all positive (i.e., virtually all pollsters overestimated Labor’s 2010 TPP performance), which produces an estimate of 45.7% ALP TPP
2. a model that is identified by assuming the house effects sum to zero (i.e., the polling industry is collective unbiased); this model produces an estimate of 47.1%.

I’ve left out the Lonergan mobile-only poll with n < 900; given that it was mobile-only, robo, I would have entered it into the model with its own unknown house effect. A pollster with an unknown house effect sees the impact of their estimate be greatly diminished, since the uncertainty generated by not knowing their house effect (and having only one poll with which to estimate it) winds up dramatically downweighting the effect of the poll on the overall estimate. Since this was a poll with a reasonably small sample size, it would have had a very small impact on the results in any event. I'll go ahead and re-run the models now with it in, but it won't change much at all, I should think.

Morgan multi-mode at 46.5 is interesting. Morgan multi-mode was showing a house effect of ALP +2, but then the last poll comes in very very close to the model consensus, dragging down the model consensus (even with some discount terms in the model) a little. There'll be time for "drop one pollster at a time" sensitivity analyses after the election, etc.

Update (7.38am AEST): it turns out that Morgan’s final numbers differed slightly from those I had in the analysis, same TPP estimate, just bigger stated sample size and the field period closed Sept 6. Re-running…

Update (10.31am AEST): No change. Lonergan and fixed Morgan field period and sample size leave model estimate at 46.2% ALP TPP. See the graph above.

## Thursday May 30, 2013

Filed under: Australian Politics,R,statistics — jackman @ 1:04 pm

I’ll be contributing a piece about once a week for the Guardian Australia, under a part of the web site we’re calling The Swing.

The set of graphs from my 1st effort were rendered in-line and rather low-res.

Bigger, full res versions appear below; click on the in-line versions.

It would be great to find a way to quickly make nice, web-friendly graphs out of R. Vega looks like a reasonable wrapper to d3. Datawrapper.de just doesn’t give me enough control over annotations, axes etc… I’m also looking at Rickshaw. Life is short, beautiful graphics are hard, sometimes…

## Friday May 10, 2013

Filed under: statistics — jackman @ 4:25 pm

I just spent a couple of hours at the meeting of the West Coast Experiments group, a set of political scientists interested in using experiments.

One the speakers was talking about the need for “credible” or “honest” p-values. “This will be good”, I thought to myself…

What the speaker was alluding to are the current moves afoot in political science to help stamp out “fishing” for statistically significant results, including pre-registering research plans. The problem is that after you’ve looked at a data set multiple times, the p-values aren’t telling what you think they are. The problem – as some Bayesians would point out – is that a p-value isn’t ever what you’d like it to be, even when you’re looking at the data the 1st time…

From the Bayesian perspective, all this stuff is kind of ridiculously overblown, a consequence of an unthinking acceptance of $$p < .05$$ as a model for scientific decision-making, point null hypothesis testing, the whole box and dice. That is worth a separate post one day.

For now, I'll remark that pre-registration of research plans is a bit like eliciting very crude priors: i.e., enumerating things to be looked at in the analysis, because the effects aren't thought to be zero; enumerating things that won't be looked at, because prior beliefs over the effects are concentrated close to zero.

The best moment of Bayesian irony was when the speaker emphasized that the need for honest p-values is especially pressing in situations where the experiment is expensive or intrusive and therefore unlikely to be run very often. This was just awesome, when you think what about a p-value is supposed to measure.

More generally, its been very interesting to bring a Bayesian perspective to my teaching about experimental design and analysis, or to a meeting like the one I was at today.

To begin with, try this on: the role of randomization in Bayesian inference. At least as a formal matter, randomization plays no formal role in the Bayesian analysis of data from an experiment, or any other data for that matter. This sounds so odd to non-Bayesians at first, particularly people who are doing a lot of experiments. But recall that repeated sampling properties like unbiasedness just aren’t the 1st or 2nd or even 3rd thing you consider in the Bayesian approach.

So just is the value of randomization to a Bayesian? Surely not zero, right? Don Rubin has written a little on this; Rubin’s point – that randomization limits the sensitivity of a Bayesian analysis to modeling assumptions – is a stronger conclusion than it first sounds, and one more of the more helpful things I’ve come across on the topic. I also found this note by J. Ghosh, a very concise and accessible summary of the issues too, summarizing some of the Bayesian thinking on the matter (Savage, Kadane, Berry etc). But my sense is that there’s not a lot out there on this. There is actually more writing on this in the literature putting model-based inference up against design-based inference in the sampling literature, which is essentially a parallel debate.

So, vast chunks of the (overwhelmingly classical/frequentist) literature on the analysis of experiments can seem very odd to a Bayesian. Randomization inference, or permutation tests. Re-randomization of assignment status if one detects imbalance. Virtually all Bayesians take the Likelihood Principle seriously, but so much of the work on experiments seems to violate it. It is also pretty obvious that experimenters are also carrying around prior information and using it: balance checks would seem to be guided by prior expectations as to likely confounders, no? Just in the same way that post-stratification weighting for non-response in a survey setting seems to be guided by an (implicit, and rather simplistic) model of response/non-response.

There is a lot to work through. Above all, it is important to keep in mind what is relevant for the applied scientist, what is more esoteric, and where Bayesian ideas can be of real practical use (e.g., Andy Gelman et al on hierarchical models for multiple comparison problems, or in the analysis of blocked or clustered designs, etc).

For now, I’m blessed to have colleagues like Persi Diaconis, Guido Imbens and Doug Rivers, who indulge (or encourage) my thinking out loud on these matters.

## Monday April 1, 2013

Filed under: politics,statistics — jackman @ 12:18 pm

“I needed to learn how to be persuasive. I needed to learn how to win arguments. And so I did two things.”

“I took a ton of statistics classes…”

“And I enrolled in the Ethics in Society Honors program.”

## Wednesday March 20, 2013

Filed under: ANES,politics,statistics — jackman @ 12:17 pm

The Senate just adopted Coburn’s (amended) amendment:

To prohibit the use of funds to carry out the functions of the Political Science Program in the Division of Social and Economic Sciences of the Directorate for Social, Behavioral, and Economic Sciences of the National Science Foundation, except for research projects that the Director of the National Science Foundation certifies as promoting national security or the economic interests of the United States.

Barbara Mikulski accepted the terms of the amendment, let it go through on the voices.

Nicely wedged, nicely played.

A great day for IR. A bad day for the study of American politics, political methodology…etc.

The “national security” or “economic interests” tests will be an interesting thing to see play out.

Or maybe we’re all sociologists now?

## Wednesday January 2, 2013

Filed under: politics,statistics — jackman @ 7:06 am

Here are two quick looks at the roll call last night from the House of Representatives, concurring with the Senate amendments to HR8. This was the so-called Fiscal Cliff vote, roll call 659 of the 2nd session of the 112th U.S. House of Representatives.

I’ve plotted the Ayes and Noes against each representative’s “ideal point”, a summary measure of each representative’s voting history based on previous roll calls in the 112th House, usually interpreted as an estimates of each legislator’s “left-right” or liberal/conservative ideological position. Democrats are shown in blue (and cluster to the left of the graph); Republicans are in red, on the right.

Notably, Speaker Boehner voted Aye, just the 9th time he has recorded a vote in the 112th Congress, which is not unusual for Speakers. With such a short, a largely one-sided short voting history, the algorithm puts Boehner out on the right-hand tail of the ideal point distribution. Majority Leader Cantor voted Nay.

The 1st graph shows a probit curve overlaid on the points, an estimate of the probability that a legislator occupying a particular point on the left-right continuum votes for the measure. Since all but 16 Democrats voted for the measure, the curve falls as we move from left to right across the page.

The 2nd graph fits probit curves separately for each party. The interesting action is among the Republicans, with the vote cleaving Republicans 151 – 85, and largely along ideological grounds. The ideal point estimate predicts Republican voting on this measure reasonably well (AUC = .816, Brier = .166).

In addition to being a rare instance of the Speaker casting a roll call vote, it is also an even rarer case of the Speaker voting against the majority of his party, including the majority leader. It might be a nice exercise to see when this last happened. We’ll see if this costs Boehner his job as Speaker; I suspect not.

## Sunday November 11, 2012

Filed under: politics,statistics — jackman @ 1:06 pm

Obama won the 2012 election by keeping the swing small where it needed to be kept small. Just two states changed hands: Indiana and North Carolina, both won narrowly by Obama in 2008.

But consider this. Florida, Ohio and Virginia were all won by Obama in 2008 on margins smaller than the 53.4-46.6 national margin. Obama kept them all in 2012, with swings smaller than the national swing. Details…

Nationally, the two-party swing against Obama is currently -2.11 percentage points: that is, Obama beat McCain in 2008 53.4-46.6; at this stage of the 2012 count he is winning 51.3-48.7.

Let’s look at some key swing states.

• Ohio (18 Electoral votes): Obama won it 52-48 in 2008 (in two-party terms). A two percent swing and Romney wins it. The swing is only -1.12, about half the national swing. Obama wins Ohio 51-49.
• Florida (29 EVs): Obama won 51.3-48.7 in 2008. A 1.3 percent swing against Obama would see the state fall to Romney. The swing appears to be -0.83 points with Obama winning 50.4-49.6.
• Virginia (13 EVs): Won by Obama 53.2-47.8. Romney needed a -3.2 percentage point swing against Obama to win the state. He got half way, -1.6 points, and the state stays with Obama.
• North Carolina (15 EVs): Obama outpolled McCain by 13,700 votes in 2008, to win 50.2-49.8. The swing was -1.26 points and Romney wins 51.1 of the two party vote.

And on it goes. Without more of these easier states coming over the line, getting to 270 was a very difficult proposition for Romney. The Romney campaign would have needed some large swings in other battlegrounds to get close.

• Iowa: Obama had a 4.7 point margin, the swing to Romney was 1.8.
• New Hampshire: 4.8 margin, a 1.9 swing.
• Colorado: a 4.4 point margin, a 2 point swing.
• Wisconsin: a 7 point margin, a 3.6 point swing (larger than the national average, and larger than the swing-state average swing, thanks Paul Ryan?).
• Nevada: a 6.3 point margin, a 3 point swing.

And Pennsylvania? Forget it. With 20 Electoral College votes you can understand why it is an enticing target. But Pennsylvania was higher up the tree than Iowa, New Hampshire or Colorado, breaking 55.2-44.8 in 2008. Romney won a 2.6 point swing in Pennsylvania (slightly higher than the nationwide swing), but still only half of what he would have needed to win the state.

Indiana was the other Romney pickup. Obama’s 50.5-49.5 victory there in 2008 was a shocker, coming on the back of 10.9 point swing: to understand how big that was, recall that the national two-party swing to Obama was 4.8 points in 2008. Indiana was never on anyone’s battleground list in 2012. Expectations were that Indiana would “revert to type”. Absent any serious attempt to defend the narrow and unexpected 2008 win from the Obama camp, Indiana swung 5.8 points in 2012 (more than twice the national swing) with Romney winning 55.3-44.7.

A similar logic applies to Missouri, which Obama almost won in 2008, falling short by 0.06 of a percentage point, outpolled by McCain by just 3,600 votes. Missouri “reverted to type” in 2012 too, with a 4.8 point swing. In fact, Obama’s 2012 showing is worse than Kerry’s in 2004 (45.1% of the two party vote, versus Kerry’s 46.3%).

Two graphs summarize all this; clicking on each will open a larger version in a new browser window.

The first is a dot plot showing the swings in order. The color shows the disposition of the state in 2008. Two vertical bars show the national swing and no swing. Open plotting symbols for swing states show the swing they would have required to change hands: for North Carolina, the open plotting symbol lies to the right of the actual swing (the swing exceeded Obama’s 2008 margin). Again, note how the swings against Obama come up short in the other battleground states.

Second, a plot of 2012 swing against 2008 result. Shaded regions denote politically distinct regions: (1) Obama retains, where the 2012 swing isn’t enough for the state to fall to Romney, with 27 states in this category; (2) Obama losses, where the swing against Obama is enough for the state to fall to Romney (Indiana and North Carolina); (3) Obama gains (a null set in 2012); and (4) Republican retains, with 22 states in this category. The District of Columbia is excluded from this graph. The size of the plotting symbol is proportional to the state’s Electoral College votes; color indicates 2012 disposition; the national result is shown with a dark point labelled “USA”.

Obama wins the election on the back of that set of states that sit just to the right of the 50-50 threshold on the horizontal axis, but that didn’t drift down on the vertical “swing” dimension into the “change hands” cone.

## Saturday November 10, 2012

Filed under: politics,statistics — jackman @ 1:59 pm

I tweeted earlier that the correlation between 2012 and 2008 state-level, Democratic, two-party vote shares is .982. Take out (the outlier) DC and this goes down to .976.

Here’s another look at the data, comparing 04, 08 and 12. More on this later (as well as overlaying the national two-party vote shares), but we have stories that go along with Alaska, Hawaii, New Jersey (!), Utah, over these three elections.

What is striking is how the swing stayed small where Obama and Co. needed it to stay small. Big swings against them in Indiana and the South didn’t matter.

## Thursday November 8, 2012

Filed under: politics,statistics — jackman @ 5:33 pm

I’ve had numerous requests for various outputs from my model, so I’ll put them up here.

1. Forecast win probabilities for Obama, by unit, by day. These were the quantities also requested by a group of researchers working at Penn, who will use Brier scoring to assess various probabilistic forecasts of the state-by-state outcomes.

A summary of how well the final (Election morning) state-by-state probabilities performed is shown in the attached graph, which plots Obama 2-party vote share (using data from this afternoon from AP) against the probabilities from my model.

The probabilities are the data I’ve presented in this graph:

The data themselves are here (CSV). The final set of probabilities also appear in the file I link to below.

2. State level predictions, Obama 2-party vote share, plus uncertainty assessments. The comparison with actual state-level outcomes appears in the following scatterplot (the diagonal line is a 45 degree line).

Data here (CSV). One of the columns is the final probability of Obama win, if you are looking for that.

## Monday October 15, 2012

Filed under: politics,statistics — jackman @ 6:52 pm

Re-posting what just appeared on HuffPost:

On the eve of the 2nd presidential debate, it is helpful to take stock of what current polling is indicating about the campaign.

Romney’s debate bounce – or rather, Obama’s debate slump – appears to be over. National-level polling appears to have bottomed out for Obama. Further, once we integrate the information about the national trend contained in the state-level polls, it appears that Obama’s post-debate slump was not as large as it might have seemed at the time (and I’m talking to you, Andrew Sullivan). Indeed, with the benefit of only a little hindsight (and only a little modeling) it appears that only about 1.5 percentage points separate Obama’s late September, national peak from his early October trough, with a recovery of a few tenths of a percentage point over the last 7-10 days.

Indeed, at the national level, it appears that even in the immediate aftermath of the 1st debate, Obama never trailed Romney, once the information in the state level polls is integrated with the national-level polling.

All the same, the national race is unquestionably tighter now than it was. Obama’s lead in the national track produced by my model is about half a percentage point. In late September this lead was four percentage points.

The swing state picture has generally remained better for Obama. In the picture below I look at the the national-level estimate of Obama’s two-party share, along with results from 9 battleground states, all won by Obama in 2008. A useful reference point is Obama’s two-party share in 2008, appearing on the left of the graph. In the middle of the graph I plot the estimates from my model using polls fielded prior to the 1st debate.

The general fall in Obama support between 2008 and where the campaign stood on the eve of the 1st debate is clear (the pattern of lines sloping down, left to right). Only in Ohio was Obama generally outperforming his 2008 result.

Since the debate, Obama’s national vote share has slipped by roughly the same amount it had fallen between the 2008 election and where it stood on the eve of the debate: from 53.7% to 52.2% (pre-debate) to 50.4% (now). Post-debate slumps of a similar magnitude appear across the nine battleground states on the graph. North Carolina now lies on 48.5% (Obama’s share of two-party vote intentions). Florida had been resisting the national trend away from Obama prior to the debate, but has also fallen below 50%. Virginia is at 49.9%. At this stage, only Indiana, North Carolina and Florida look like they will be picked up by Romney, which won’t win Romney the election.

Obama won Ohio with 52.3% of the two-party vote in 2008. Pre-debate, I estimated Obama two-party support there at 53.4%. Today that figure is 51.5%, with the falls relative to both the 2008 result and the pre-debate estimate the lowest among the swing states. Relative to 2008, Obama is off 0.8 percentage points in Ohio; the corresponding numbers are 1.7 in North Carolina, 2.1 in Florida, 2.9 in Iowa, 3.3 in Virginia (the same as the national-level swing), 3.9 in Colorado, 5.1 in Wisconsin, and 5.3 in Nevada.

So long as Obama’s Ohio numbers hold up, so too will his chances of winning the Electoral College, and thus far, they have. Music to the ears of owners of Ohio’s television stations…