Mixed treatment counterfactual Obama

While I was working on something completely unrelated to this, last Friday I came across a paper by Simon Jackman; I've known of his work for a few years (for example, I think his book is quite good) and I quite like the fact he's applying high level Bayesian stats to political science, something I'm interested too.

This paper (number 1 in the list here) is quite thought provoking, I thought, because it mixed a few topics I've been thinking about lately. In a nutshell, they are trying to estimate the "Obama effect" in the 2008 US election. The starting question is "Would the outcome of the 2008 U.S. presidential election have been different if Barack Obama had not been the Democratic nominee?" Now, this obviously has a counterfactual flavour; of course, Obama was the democratic nominee and thus (if, like me, you're that kind of person) you may think that the question does not even make too much sense, in the first place. 

But this is slightly different than usual causal analysis (usually based on counterfactual arguments); first of all, the "Obama effect" (mainly defined in terms of the president's race and its impact on the US electorate) does matter both for Obama himself, in the next election, as well as for similar situations in other elections. In addition, the analysis is based on a standard hierarchical model (ie it does not use potential outcomes) and the authors are extremely clear and upfront about the assumptions and limitations in terms of their causal conclusions, which I think is really good.

The study is based on a mixture of external data and specific surveys in which the authors tried to assess the sample's preference for different, hypothetical head-to-head comparisons. For example, before the 2008 election, they repeatedly surveyed US voters asking who would they vote for if the next election involved different combinations of Democratic-Republican candidates (Bill and Hillary Clinton, Obama, Edwards and Gore vs Giuliani, Bush, Huckabee and Romney).

Effectively the design is based on mixed treatment comparisons, as not all the samples were given the choice among all the possible head-to-head pairs. This is quite interesting and is quite straightforward to deal with using Bayesian hierarchical models $-$ if some exchangeability assumptions hold, then different comparisons can inform each other, so that all the possible comparisons can be estimated (of course comparisons informed by less hard evidence will be associated with larger uncertainty, but that's entirely reasonable).