Marking your own homework

I quite like the way in which Brian Leveson (who has led the famous public inquiry into the media, in the UK) has summarised his recommendations: you guys [ie the press/media] should not "mark your own homework".

I'm afraid that's exactly what the Prime Minister Kirk Cameron will allow. By the way: isn't it kind of neat that in the picture I've put here he quite looks like vice-PM Nick Clegg, instead? 

Perhaps, despite the apparent argument the pair are having about whether the over 2000 pages Leveson inquiry report should be used to replace the use of Cushelle $-$ which incidentally would free us from the unbelievably stupid advert they have $-$ they are actually morphing into a single person!

And also, if that's how it goes on, may be we should consider letting our students marking their own homework; it would come quite handy, given that I have a nice pile to mark right in front of me...

Grant (and missing data)

Today has been quite an interesting day. First of all, we finally heard from the MRC and we got the Grant! (I actually mean money to do our research on the regression discontinuity design, not fellow Fulham FC fan Hugh $-$ but he arguably looks better than a graph with dots and lines...).

We had put quite a lot of work on the write up (actually since the moment Sara pitched the idea and we started discussing it), especially as we were really close to be funded, when we first submitted it last year. The panel liked the idea but asked us to change a few things and re-submit it, which we did. This time around we've been luckier and I'm really pleased. The research group is fantastic and I'm really looking forward to starting it!

As for the rest of the day, I spent it in seminars/workshops, mainly about missing data. In the morning I went to one of our monthly seminars with the PRIMENT people; today's talk presented the statistical plan for a trial that they are developing. Some of the discussion was on how to deal with the expected, relatively large proportion of missing data.  

This linked nicely with the LSE workshop I went to in the afternoon (I nearly managed to make it on time for lunch, but as it turned out, I got there a bit too late, so I ended up not eating). The focus of the workshop was on linking survey weights and methods for missing data (specifically multiple imputation); this is interesting as I'm trying to include missing data in my revised lectures for Social Statistics (which will be in the next term). 

The perks (and quirks) of being a referee

The other day I was talking to a friend at work, who was rather annoyed that one of his papers had been rejected by a journal, given the negative comments of the reviewers. This is, of course, part of the game, so you don't really get annoyed just because a paper get rejected. From what I hear, though, I think my friend was quite right in being angry.

The paper was submitted to a medical journal; the editor had sent it out for review to 3 referees, two of whom were, allegedly, statistical experts. I hadn't read the paper, nor the reviews, so I can't comment in great details. But from what I hear, the reviewers' comments were just wrong. In practice, they told off the authors for using wrong statistical methods, while it looks like they just didn't understand the valid statistical point.

For example, one of the referees had criticised the following: for some reasons (I can't remember the details), the authors had performed a regression model and then regressed the resulting linear predictor on the covariates, which obviously leads to $R^2=1$. 

Now, you can certainly debate as to whether the methods used by the authors were the most appropriate for their purpose, but their point was not wrong $-$ you can easily check in R with the following commands

# Simulates some covariates
x1 <- rnorm(100,0,1)
x2 <- rpois(100,2)
x3 <- rbinom(100,1,.6)
#(Arbitrarily) sets the coefficients for each covariate
beta <- c(1.43,-.14,.97,1.1)
# Computes the "true" linear predictor
mu <- cbind(rep(1,100),x1,x2,x3)%*%beta
# (Arbitrarily) sets the population standard deviation
sigma <- 2.198
# Simulates the response
y <- rnorm(100,mu,sigma)

# Fits a linear regression & show the results
m <- lm(y~x1+x2+x3)
summary(m)

Call:
lm(formula = y ~ x1 + x2 + x3)

Residuals
    Min      1Q  Median      3Q     Max
-5.0186 -1.4885 -0.0434  1.4007  5.7971

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  1.86280    0.50344   3.700 0.000359 ***
x1          -0.03908    0.24307  -0.161 0.872618
x2           1.05753    0.15927   6.640 1.88e-09 ***
x3           0.41025    0.45461   0.902 0.369090
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.244 on 96 degrees of freedom
Multiple R-squared: 0.3154, Adjusted R-squared: 0.294
F-statistic: 14.74 on 3 and 96 DF,  p-value: 5.692e-08

Of course, because of sampling variability, the coefficients are estimated with error; in addition, the overall model fit (as measured by $R^2$) is not perfect, with only 32% of the total variability explained by the regression. If however we regress the fitted values on the same set of covariates:

m1 <- lm(m$fitted.values~x1+x2+x3)
summary(m1)

Call:
lm(formula = m$fitted.values ~ x1 + x2 + x3)

Residuals:
     Min         1Q     Median         3Q        Max 
-1.560e-15 -2.553e-16 -1.035e-17  2.161e-16  2.699e-15

Coefficients:
              Estimate Std. Error    t value Pr(>|t|) 
(Intercept)  1.863e+00  1.193e-16  1.562e+16   <2e-16 ***
x1          -3.908e-02  5.758e-17 -6.786e+14   <2e-16 ***
x2           1.058e+00  3.773e-17  2.803e+16   <2e-16 ***
x3           4.103e-01  1.077e-16  3.809e+15   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 5.315e-16 on 96 degrees of freedom
Multiple R-squared:     1, Adjusted R-squared:     1 
F-statistic: 2.627e+32 on 3 and 96 DF,  p-value: < 2.2e-16 

this now implies perfect fit $-$ but that just makes sense as the linear predictor is given by exactly that combination of covariates.

I'm not defending my friend's paper for the sake of it $-$ to reiterate, I haven't read it and I don't really know whether it should have got published. And maybe there were other issues that the reviewers rightly picked up. But certainly it is wrong that it was judged as statistically flawed, and I think I would probably write a response letter to the editor to argue my case.

Of course this is a very delicate issue, and people often voice their strong opinions about the state of peer-reviewing; Larry Wasserman even goes as far as to argue that we should completely dispense with them. 

No junk mail, please

The last two comments I received (on this post) were quite odd. Somebody has left a comment saying something like "Thanks! Good to see an update" and then a link to some random page advertising some stuff that had absolutely nothing to do with the content of the original post.

I decided to let the first instance go, although I was a bit annoyed. This morning I found a second comment by the same person, and so I took action and deleted both. I have also sent the guy an email to explain that I don't think this is cool, but I doubt he'll ever actually read it (or care).

(Nearly) sold out

The other day I have spoken with the publisher who told me that the book is officially out in the US; usually it takes a few weeks to stock in Europe, so it'll probably be available here by early December. Obviously, this is all very exciting (well, at least for me). 

So I checked how it was doing on amazon.com. Apparently, there are only 2 more copies left! I don't know if they had only stocked 3 copies and they have sold 1 since the book was out earlier this week, but thankfully they say that there are more on the way...

Porn economics?

Since it's Saturday and Marta is on holiday at IKEA with her mum, this morning I took it really easy [NB: wait until you read the whole thing, before you judge me from the title and the premise $-$ it's definitely not what it looks like!]. 

I had spot of breakfast, shaved, showered and then took a look at the newspapers. While I was reading the Guardian, I saw this article. I didn't know anything about this, but the gist of the story is that in addition to re-electing President Obama, earlier in November the people of California have also voted in favour of Measure B, a new law requiring the use of condoms in adult movies shot in LA County. 

The author of the article is Stoya, a "performer for adult production studio Digital Playground" [I thought of posting a link, but from her own profile in the Guardian's website, she "recommends that you refrain from googling her at work", and so I thought better not]. Her argument against the new law is that the porn industry has not seen a single case of performer-to-performer HIV transmission in the last 8 year (which I suppose it's impressive). On the other hand, there is some evidence that movies in which the performers wear condoms are less well received by the audience, leading to a drop in sales. This, she continues, will have an effect on the whole industry; at the moment, the performers are continuously tested for STDs (including HIV), which of course is quite expensive. Thus, if the industry's profits are reduced further (in addition to the losses inflicted by piracy, mostly on the web), this will lead to fewer performers being tested and thus potentially producing unintended negative effects.

I think that Stoya is making a good job at trying to argue from an economic perspective (with this I mean that she's not making it all about the money, but also the intended and unintended consequences of interventions). For example, she says that condoms can cause abrasions during the "abnormal" [her words] sex-making sessions in porn movies; thus, if a condom fails, this can even increase the risk of disease transmission.

That's interesting: I am not entirely convinced of the underlying (informal) model she is proposing, as I think she's possibly leaving other factors out of the equation, that should be taken into account. For example, I've no idea about this, but surely there must be statistics on other performer-to-performer infections; what if these show rates greater than 0? Sure, HIV is probably the worst outcome, but other diseases (for example HPV) may be also very relevant, both from the health and the financial perspective. 

And then there is the "educational" issue: given the large number of people watching porn, perhaps there is an explicit utility in showing performers wearing condoms. I absolutely have no idea or evidence that I could easily access here, but could this be even stretched as far as to say that it is cost-effective to invest public money in helping companies implementing this policy? 

Finally, some of the comments to the article, suggest that one of the obvious implications will be that some of the productions will probably move away from California (or LA, at least). This reminds me of a kind of similar issue with financial banks, here in London. The argument against increasing their fiscal contribution to the UK government is that in that case the corporations may choose to leave for countries with lighter taxations. Again, I think this is just one side of the story, as there is an implicit utility to being based in London (or in LA, if you're an actor/producer). While I understand that moving a business somewhere else could lead to losses to the central government (LA county in this case), I also think that this is not an entirely genuine threat. 

May be the implementation of the policy should have been subject to the results of a full cost-effectiveness analysis (I doubt it was $-$ but that's just a very uninformed opinion). I wonder if the NIH (or someone like them) would fund someone to do it?

When in Rome...

Yesterday I was in Rome to teach in a short course on Bayesian methods in health economics. 

The 6.45am flight from London was actually on time, which was impressive, considering that the last time I flew Alitalia I never made it to Rome $-$ we stopped in Milan but, because of "technical problems", the flight was cancelled. I had to give a talk via Skype from the airport and then got back to London with the very last flight, although I was originally supposed to be back with the 7.30pm flight. 

I arrived at Fiumicino at about 10.30am and after the passport control I headed to the train station. Unfortunately, there was no train scheduled to go into central Rome in the near future (or, for that matters, even in the distant future, according to the electronic board). So I walked back to the coach station. Signs on either side and on the front of the coach, as well as on the actual ticket said €4 one-way. 

The driver however said that it was €5. 

After a few minutes we left. Before we were even out of the parking lot, the driver was already shouting on his mobile phone (the first call was to "Fra") $-$ needless to say, he did not have blue-tooth or headphones; although to be fair he was remarkably good at handling the steering wheel with his knees. 

The first part of the journey into Rome is on the motorway and it wasn't very busy, so he was just happy to chat away, mostly boosting to his friends that they got stuck in traffic because they didn't ask his advice; but there also was a call to his mum (who, apparently, had annoyingly failed to talk to dad, which means he would have to do it himself) and a failed attempt to contact his wife, Barbara.

When we actually got to the outskirts of town and off the motorway, the traffic became a bit worse. At that point, his friend "Fra" called again, and again the driver started to playfully (I think) insult them because they were stuck in traffic. Until we also got stuck in a big jam, that is. In response to this, he started to honk the horn and shout at everybody, including a policeman who was handling the traffic (well, he wasn't really doing a good job, but still...).

Finally, a good 90 minutes after we left the airport, we arrived in central Rome, which unfortunately was still not where I needed to go. Because of a strike and a couple of demonstrations, the traffic in the area was mental. There was no official queue for the taxi, but I still managed to waive at and stop one, so I suppose I shouldn't complain too much that to do this (6km) it took me another 45 minutes.

After a quick lunch, we started the course; the turnout was all right (about 20 people) and I think it went reasonably well $-$ although, as I suspected, we had planned a bit too much. I have given this lecture a few times in the last months (although in slightly different formats and this time it included more bits about the general principles of Bayesian statistics) and there are a couple of things that I think are really interesting. 

The first is that people seem to be genuinely surprised to hear about the controversy between Neyman-Pearson and Fisher and that they couldn't even bear to be in the same department (the usual reaction from the audience is to think I'm joking). The second is the reaction to my point that the prior information and the prior distribution are two different things, which I always stress. I think people generally take this well and I think it makes it a bit easier to come to terms with the idea of formulating the prior as just one possible probabilistic "translation" of some knowledge, which can be generally expressed in words.

At the end of the course, I took a taxi to the airport. It was still pretty busy, but that didn't bother me that much (by then I'd sort of given up). I got to the airport in time for a quick (and incredibly expensive) sandwich before boarding the flight, only to discover that they had assigned seat 4A to 4 people $-$ of course I was one of them. The plane was not very full, but they still spent a good 15 minutes frantically calling I-don't-know-who on the phone to try and "sort it out". Which they did, in the end $-$ by telling three of us to just find another seat.

You can't play a broken link

Just like James Morrison and Nelly Furtado say, you really can't play a broken string. And quite similarly, you just can't use a broken link.

I always find it very annoying when, while browsing a website, I find a broken link. You fall victim to the promise of interesting things to come, only to be disappointed by the error message filling up your screen $-$ something like when you're trying to call somebody on their phone and you get the "Sorry. The number you've dialled is not recognised" message.

But, I should really stop moaning about all this because, as it turns out (thanks to semi-anonymous George from Canada), I'm guilt of this crime myself and there were some broken links on the book webpage. Some of the files with the R code for the examples in chapters 4 and 5 (which I've already discussed here and here) were pointing to the wrong addresses and therefore weren't downloadable. This should be fixed now, so hopefully it will all work.

Gotcha!

I should start this with a disclaimer, ie that I'm not really claiming any "success" with this post. But I find it quite interesting that the estimations I produced with this very, very simple model turned out to be quite good.

The idea was to use the existing polls (that was a few days ago, even before the super-storm), which had been collated and presented in terms of an estimation of the proportion of voters for either party, together with some measure of uncertainty. Based on these, I constructed informative prior distributions, which I have then propagated to estimate the election results.

As it turns out, according to the projections of the final results, the prediction was accurate, as the following graph shows: the dots and lines indicate the average prediction and a 50% (darker) and 90% (lighter) credible intervals; the crosses are the observed proportions for Obama.

In all states, the prediction was "correct" (in the sense that the right "colour" was estimated). In some cases, the observed results were a bit more extreme than the observed ones, eg in Washington (WA) the actual proportion of votes for Obama is substantially larger than predicted $-$ but this has no real consequences on the final estimation of the election results as WA was already estimated to be a safe democratic state; and this is true for all other under/over estimated cases.

My final estimation was that, based on the model, I was expecting Obama to get 304 EVs. At the moment, the Guardian is reporting 303 $-$ so pretty good!

But, as I said, this is really not to brag, but rather to reflect on the point that while the race was certainly close, it probably wasn't as close as the media made it. Famously, Nate Silver gave Obama a probability of winning the election exceeding 80%, a prediction which has given rise to some controversy $-$ but he was spot on.

Also, I think it's interesting that, at least in this case, the polls were quite representative of the "true" population and what most people said they would do was in fact very similar to what most people actually did.

Mapping in health economics

Last Friday, I went to one of the health economics seminars that are organised at UCL; the format that is used is that one of the people in the group suggests a paper (typically something that they are working on) but instead of having them leading the discussion, one of the others takes the responsibility of preparing a few slides to highlight what they think are the main points. The author/person who suggested the paper is usually in the room and they respond to the short presentation and then the discussion is open to the group at large.

I missed a couple since they started last summer, but the last two I've been to have really been interesting. Last time the main topic was mapping of utility measures; in a nutshell, the idea is that there are some more or less standardised measures of "quality of life" (QoL $-$ the most common probably being the EQ5D and the SF6D). 

However, they are not always reported. For example, you may have a trial that you want to analyse in which data have been collected on a different scale (and I'm told that there are plenty); or, and that's perhaps even more interesting, as Rachael pointed out at the seminar, sometimes you're interested in a disease area that is not quite covered by the standard QoL measures and therefore you want to derive some induced measure by what is actually observed.

In the paper that was discussed on Friday, the authors had used a Beta-Binomial regression and were claiming that the results were more reasonable than when using standard linear regression $-$ which is probably sensible, given that these measures are far from symmetrical or "normally distributed" (in fact the EQ5D is defined between $-\infty$ and 1).

I don't know much about mapping (so it is likely that what I'm about to say has been thoroughly investigated already $-$ although it didn't come out in the seminar, where people were much more clued up than I am), but this got me thinking that this is potentially a problem that one can solve using (Bayesian) hierarchical models.

The (very raw) way I see it is that effectively there are two compartments to this model: the first one (typically observed) is made by data on some non-standard QoL measure and possibly some relevant covariates; then one can think of a second compartment, which can be build separately to start with, in which the assumptions underlying the standard measure of QoL are spelt out (eg in terms of the impact of some potential covariates, or something).

The whole point, I guess, is to find a way to connecting these two compartments, for example by assuming (in a more or less confident way) that each of them is used to estimate some relevant parameter, representing some form of QoL. These in turns have to be linked in some (theory-based, I should think) way. A Bayesian approach would allow for the exchange of information and "feed-back" between the two components, which would be potentially very helpful, for example if there was a subset of individuals on which observations on both the compartment were available.

I'll try to learn more on this $-$ but I think this could be interesting...

Hand picking

Yesterday we were out with our friends; we met for a drink late in the afternoon and spent quite some time trying to figure out what we wanted to eat.

First, we approached the problem by area, trying to think where we wanted to go and then looking on the internet on our phones (well, at least those of us who have modern phones, ie not Christian or Vale), but that didn't work out.

So we decided to browse by type of food and I think it was Marta who found an Eritrean place close to where we were. Because of the bonfire, it took us nearly twice as long as normal to get there, but in the end we made it. And it was really good!

They bring you a big tray covered with a flat pancake over which they then put all the food; and the fun part is that you are supposed to pick it with your hands, eventually eating also the pancake. At the end of the dinner, they also offer a typical coffee ceremony, which we tried. It takes for ever (in all fairness, they roast and ground the coffee), but to keep you busy the also bring some popcorn, which is unusual as well as actually nice.