At the Copa

This is the first post of a fairly regular series (at least I'll try to keep it this way!), dedicated to the impending FIFA World Cup (you may think I've gone all Barry Manilow, like Peter & co $-$ but I can reassure you I haven't).

Marta, Virgilio and I have discussed this at the end of a barbecue at our place a few weeks back and have then done some work (in fact, Virgilio had prepared a different version of the model $-$ I'll mainly describe what Marta and I have worked on, but I'll also try and discuss Vir's analysis, at some point).

As I briefly mentioned in an earlier post, the main idea is to extend the model we developed to predict football results, for international games. Basically, we have considered a dataset of nearly 4,000 games, including the last six world cups (from Italia 90 onwards) and the last 4 years (including friendlies, qualifiers, continental finals and Confederation Cup). The rationale for including these games is that:

1. We wanted to have some direct evidence on World Cup games; we restricted to the last 6 WCs which all have a very similar format (with a first round of games and then a knockout stage, starting with a round of 16). 
2. We also want to have evidence on most recent performances of the teams affiliated with FIFA. 

I think it's worth noticing that many people (eg Goldman Sachs or 538) who have had a go at a similar exercise have decided to discard friendlies. But it seems to me that these may be indicative of some general trend and so we decided to keep them in our dataset.

The main outcome of our model is the number of goals scored by each team in each of the games. In particular, we want to account for the fact that the two counts are correlated. We model

$$y_i \sim \mbox{Poisson}(\theta_i)$$

where $y_i$ is the number of goals scored in game $i$. Note that we replicate the same game twice, looking at it from each of the two teams' perspective, respectively. So, for example, the first game in the dataset is Argentina-Cameroon, the opening game of Italia 90 and the first two rows in our dataset describe the game from the perspective of Argentina and Cameroon, respectively, like so:

Game	Team	Opponent	Goal	...
1	Argentina	Cameroon	0	...
1	Cameroon	Argentina	1	...

The "propensity" of the team in row $i$ to score when they play against the team in row $i+1$, which we indicate as $\theta_i$, is modelled as

$$\mbox{log}(\theta_i) = \mu + \beta_{\mbox{home}} \mbox{Home}_i + \beta_{\mbox{away}} \mbox{Away}_i + \beta_{\mbox{type}}\mbox{Type}_i + \beta_{\mbox{form}}\mbox{Form}_i + \gamma_{\mbox{team}_i} + \delta_{\mbox{oppt}_i}.$$ The linear predictor is made by:

1. The overall intercept $\mu$;
2. A set of unstructured effects $\beta$, accounting for the effect of i) playing at home, away or on neutral ground (Home and Away are dummies, so that when they are both 0, then the game is played on neutral ground); ii) the type of the game, which could be "finals" (World Cup or Continental tournament, or the Confederation Cup), or "other" (including friendlies and qualifiers); iii) the difference in recent forms between the two opponents $-$ this is computed by accounting the mean number of points obtained in the last two games played by each team. These are rescaled in the interval [0;1] so that the difference is a continuous variable defined in [-1;1] (a value of 1 indicates that a team is much more "in form" $-$ and thus potentially stronger $-$ than their opponent for that game);
3. A set of structured effects $\gamma_t$ and $\delta_t$ $-$ these are team-specific random effects, modelled as exchangeable. Effectively, they can be interpreted as "attack" (for the team) and "defence" (for the opponent) strength.

We fit a Bayesian model (using INLA to speed the computation up $-$ it runs quite smoothly and quickly; about 30 seconds on a medium range computer). 

Then, for each of the future games (ie those to be played starting this week) we can simulate from the posterior distributions of the linear predictors $\theta_i$. When opportunely rescaled, these can be used to simulate the number of goals scored in the next games by the two teams playing.

In particular, given evidence that at the World Cups usually fewer goals are scored, we have inflated the chance of seeing a 0, so that the prediction is made as
$$y^{\mbox{new}}_i \sim (1-\pi)\mbox{Poisson}(\theta^{\mbox{new}}_i)$$
where $\pi$ is given an informative distribution based on the assumption that the chance of excess 0s in the observed number of goals at the World Cup is centered around 0.035 with a standard deviation of 0.02 $-$ in actual facts, we've tried a few alternatives, but this assumption does not affect the estimation/prediction massively.

In addition, for the yet-to-be-played games, the value of the "recent form" variable (which sort of determines the difference in strength between the two teams playing in a game $-$ we indicate this as $\omega_t$ and $\omega_s$ for teams $t$ and $s$, respectively) is based on an evidence synthesis of the available odds for each of the team, rather than on the actual past 2 games. It is easy to find data on lots of bookies offering odds for each of the 32 teams involved in the WC $-$ we've based our evidence synthesis on the 20 values found here.

I think there is a good reason for doing this: the last two games observed in the dataset are friendly games played in preparation for the finals. In those games, teams tend to train really, but do not give their 100%. On the other hand, the valuations of the bookmakers should be a more reliable indication of the actual relative strength of the teams involved at the moment. 

Thus, we decided to use those values (starting from the odds, we built a simple log-Normal model and then rescaled the team-specific effects in the scale [0;1] to indicate the "recent form" of the teams).  Under this sub-model, Brazil has a score of recent form of 0.97, closely followed by Argentina with a score of 0.96. The weakest team are Costa Rica, with a score of 0.044. If you're into graphical representations for models, here's one for you.

Rather than predicting all the way to the final based on the model and data available right now, we decided to take it step by step. I think there is a very good argument to do so. In fact, it is quite likely that recent form will be modified by the games that will be played in the course of the first round. 

For example, group D (including Uruguay, Costa Rica and my personal derby of England and Italy) is arguably the closest with 3 relatively strong teams (538 seem to think so too). So, if Italy beat England on Saturday this surely will swing the odds in their favour, thus probably modifying quite massively the behaviour of the other teams in the next games (eg if England lost on Saturday, then they'll have to win their next game against Uruguay, while Italy may be happy to get a draw against Uruguay in their final game of the round, etc...).

So, in order to make more sense of the model, we'll only predict batches of games; before any game is played and based on the current model and data, we'll predict the first 16 games to be played $-$ that's when all the finalists will have played their first game. Then we will update the recent form variable based on the observed results and re-run the model to predict the next batch of 16 games. And then we'll repeat this step again and predict the final batch of 16 games for the group stage.

We'll take it from there and see how to carry the model forward into the knockout stage. I'll post some predictions, results, graphs in the next couple of days.

Enjoy the silence

I've been quite silent on the blog in the past few weeks $-$ a combination of exam-marking, conference-organisation and other few (some more, some less interesting) things...

As for Bayes Pharma, we're nearly there $-$ the conference is this week Wednesday to Friday. I've nearly got everything ready $-$ at least all I can think of, that is. We've arranged the social event (I'll post pictures, especially if the weather keeps good). The finalised programme with talks & titles is here. I'll post on how the conference progresses in the next few days.

On a totally different note, Marta and I have spent some time working on extending our football model to do some prediction for the impending World Cup. Lots of people have had a go (including some people at Goldman Sachs and others in the new Significance website $-$ eg here and here). 

Unlike many, we won't try to predict the overall winner straight away $-$ I think that there are very good arguments to not doing that: 1) the stage group may significantly differ from the second stage; especially in a competition such as the WC, which is played over one month, the impact of current (or very, very recent form) can be dramatic. 

So, what we'll do is: 

1. Use past data (on the last 4 years of international games + the last 6 World Cups, in total about 3500 games) to fit an extended version of our model (which accounts for correlated "team" and "opponent" structured effects); 
2. Use data on "current form" (based on the official bookmakers' odds) to predict the first round of games. The prediction can be assessed against the observed results which will become available in the first few games of the competition; 
3. Update the variable of "current form" based on the observed results (so if a team unexpectedly win their first game their "form" is bumped up and they should be predicted to do better than they would have been with the previous data only). These new data can be used to predict the second batch of games (and so on for the third batch of data).

Once the group stage is over, we'll carry the predictions forward.

I'll post more details on the model and the results of the prediction exercise (including how well we're doing) in the next few days.

History hour

This is kind of cool $-$ that's an article by Joseph Mazur, appeared in The Guardian with a very brief (but interesting) discussion of the origin of mathematical symbols. I had never really thought about it and basically gave the use of familiar symbols for granted. 

But, apparently, the first recorded instance of the use of the equality symbol $=$ is in Robert Recorde's book on algebra (The Whetstone of Witte), in which he says:

"Howbeit, for easy alteration of equations, I will propound a few examples, because the extraction of their roots, may the more aptly be wrought. And to avoid the tedious repetition of these words: is equal to: I will set as I do often in work use, a pair of parallels, or Gemini lines of one length, thus =====, because no 2 things, can be more equal." 

(incidentally, the equation he referred to was $14x + 15 = 71$).

RSS Young Statisticians Writing Competition

Significance and the Young Statisticians Section of the Royal Statistical Society host an annual competition to promote and encourage top-class writing about statistics. This year’s competition closes on 30 May 2014.

Here is link to an article on the Significance website, which explains the competition and the entry criteria.

This year, three finalists will be invited to present their work at a special session of the RSS Conference in Sheffield, in September, where the overall winner will be announced. As usual, the winning article will be published in Significance. 

Priorities

I like the way Channel 4 news prioritise their stories (that's their homepage as of 12.06PM $-$ below the one about Mrs President Obama, but above the one about Mr President Putin...

UCL Biostats Network Symposium 3

We've finalised the line up for the next UCL Biostatistics Network Symposium on "Contemporary Statistical Methods in Medical Research".

This year we'll have two sessions:
1. Risk prediction models  in Health research 
$-$ Ian White, MRC Biostatistics Unit, Cambridge 
$-$ Ewout Steyerberg, University Medical Center, Rotterdam
$-$ Menelaos Pavlou, UCL Statistical Science

2. What you cannot do with observational data
$-$ Irene Petersen, UCL Primary Care and Population Health 
$-$ Frank Windmeijer, Centre for Market and Public Organisation, University of Bristol
$-$ Aidan O'Keeffe, UCL Statistical Science

Not without some controversy (you might have seen this if you follow the allstat mailing list), the symposium (by the way, I think the PhDComics cartoon on how you should call an academic event is awesome!) is free for UCL members.

Non UCL members will have to pay a (I think very reasonable) fee of £25 $-$ details here.

In the media

Yesterday, UCL News Office issued this press release which mentions our (that's Marta and me) paper on the Eurovision contest, which has just been published in the Journal of Applied Statistics.

The idea of the paper was to try and quantify the presence of "bias" in the votes, as is sometimes (in fact quite often) suggested in the media. 

In the model, we have "controlled" for some contextual and act-specific related factors (such as the sex of the performer and the language of the song), but really what we wanted to model was the "residual", structured effect, which could explain (at least partially) why two performers sharing common features get different scores from different voters.

Basically we modelled a structured effect $\alpha_{vp}$ for each combination of voters ($v$) and performers ($p$) over the repeated measurements in the last 20 years or so. This is defined as a function of three main factors:

1. A "geographical" effect $\psi$, which accounts for potential bias due to spatial proximity;
2. A "cultural" effect $\delta_{{R_v}p}$, which we use to describe the underlying "clustering" of voters in a set of groups $-$ we term this "cultural" in a rather broad sense. The clustering estimation is embedded in the model;
3. A "migration" effect $\phi$, which accounts for the fact that voters where a large population originally from the performer's country may inflate the voting pattern in their favour.

As it happens, our model doesn't seem to uncover any evidence of negative bias $-$ no systematic low scores for any given performer $-$ although there is some (rather weak) evidence of positive bias, eg some countries showing higher propensity to score some performers higher than others.

By "standardising" the effects $\alpha_{vp}$ (centering them around their mean and re-scaling in terms of their standard deviation), we can compare them. This is the graph of propensity to vote for the UK.

We arbitrarily (but reasonably, as the $\alpha_{vp}$ are pretty much Normally distributed in their posteriors) set the threshold for "substantial" bias at $\pm$ 1.96. As you can see, while a couple of voters (Ireland, Malta and Italy) show on average a positive attitude towards the UK acts, nothing is really going on $-$ basically all credible intervals intersect 0. Most importantly, none is above or below the "bias thresholds".

We're starting to get some coverage (in some cases from the most unexpected sources!) $-$ some examples are here, here. I've also been called on the phone by The Sun (I was quite nervous about it $-$ hopefully I haven't embarrassed myself...)

On demand (but on a very serious topic)

My friend Virgilio has posted this on his Facebook page and invited me to comment. It is an article by a Spanish cardiologist that tells the story of a patient who has suffered a second stroke in a short amount of time $-$ as it turns out, the medication he was supposed to be taking was far too expensive for him, so he stopped the treatment, which presumably led to his second stroke.

Of course the individual story of a single, unemployed patient who cannot afford treatment is tragic. And of course, as I understand it, the general situation in Spain is not OK at the moment and this is likely to not be an isolated case, alas. 

But I still think that the whole rationale of optimizing health resources allocation so that the overall population, on average, benefit from them, makes sense. Of course, how decisions (eg about pricing) are actually made, it's another matter and I'm not too familiar with the details of the Spanish system to comment on that. 

Also, of course as I said that one story is tragic. But reporting bias may be present here: can we discount the possibility that many other cases do benefit from the fact that some form of prioritisation is made? For example, public money is probably saved from not giving treatments for other disease on the public health service, This means that (OK: probably not enough money, but at least some) money can be allocated on this area.