As you know, this site makes use of the fantastic football statistics put together by Opta and delivered by the Stats Zone iPhone app. I am hugely in awe that while I sit on my sofa watching a match I can receive near-live statistics on each pass, shot and tackle taking place for less than the cost of a pint per season. This is the future and I’m glad to be a part of it. In fact, without Colm McMullan’s Total Football app (the precursor to Stats Zone), this website would probably not exist.
This increase in the availability of match statistics makes analysing football matches much easier, and has given rise to a number of armchair pundits such as myself. For example, want to prove who is the best goalkeeper? The shots saved percentages are right there. Which player is best at keeping possession? Just look at the pass completion percentages. It couldn’t be more simple with the facts available to all.
Unfortunately, it isn’t that simple. There is a reason why statistical analysis underpins most of science. Physicists at the Large Hadron Collider at CERN recently announced that they have effectively found the Higgs boson, but it will take a year to prove that the statistics suggesting it exists are genuine and that all other potential explanations have been ruled out. They are currently doing the same thing with the statistics “proving” that neutrinos travelled faster than the speed of light. Don’t worry, I won’t mention nuclear physics again, nor will I suggest that this rigour needs to be applied to Christopher Samba’s heading stats, but it illustrates how large and important a field statistical analysis is.
To take an example from a discussion I recently had on twitter: it was quoted that “Brad Friedel is far outperforming [Wojciech] Szczesny” as their shots saved percentages are 79% and 61% respectively. I believe that it is impossible to draw that conclusion from those statistics for reasons I will shortly explain, although before that I should note for full disclosure that I am an Arsenal fan, even though I would make exactly the same argument had the statistics been the other way around.
The reason why that conclusion is impossible to make (or to put it another way, why that statistic is meaningless as a method of evaluating goalkeepers) is because it suffers from a lack of context. Basically, it assumes that all shots are equal, when there is a very simple thought experiment which disproves this:
Picture two goalkeepers trialling for a place in a team you manage. You put them in a match on opposite sides and sit back to evaluate their performances. Team A has a midfield full of ‘Charlie Adam’s, constantly taking shots from 40 yards out which bobble towards goal. They get ten of these shots on target, and score three goals. Meanwhile Team B has wingers like Bale and Valencia, constantly firing dangerous crosses down the corridor of uncertainty for their strikers to shoot from six yards out. They also get ten of these shots on target and score seven goals. Which goalkeeper do you select?
Team B’s goalkeeper only let in three goals, while Team A’s let in seven. Team B’s shots saved percentage was 70%, while Team A’s was only 30%. However, I’d pick Team A’s goalkeeper all day long. Why? Because based on the context of the shots he had to face he wouldn’t be expected to save any of those point blank shots but he managed to keep three of them out, while Team B’s keeper should have thrown his cap on all ten of those weak long range shots but he somehow managed to let in three of them.
This is all very theoretical, so here’s an example from the games played on December 27th:
As you can see, Friedel faced two shots, saved them both, so had a clean sheet and a shots saved percentage of 100%. Szczesny on the other hand only faced one shot, let it in so had a shots saved percentage of 0%. Pretty clear cut who was the better keeper, no?
Well, no. Friedel faced two shots from outside the area, both of which he would be expected to save, and that’s exactly what he did. The goal Szczesny conceded was a deflected shot which Fletcher expertly steered back across goal into the far corner, wrongfooting the keeper, and realistically no goalkeeper would have been expected to keep that chance out. I’m assuming you’ve seen the goal but if not try the ESPN Goals app or Eurosport highlights:
The point is, 100% shots saved compared to 0% shots saved seems pretty damning, but when context is added, both percentages would be expected given the shots each keeper had to face, so using those statistics to compare the keepers is impossible. Note that I am not arguing that Szczesny is better than Friedel; this is not a defensive post and in my own personal opinion both are two of the best goalkeepers in the Premier League. All this post is claiming is that it is impossible to draw conclusions about goalkeepers based on shots saved percentages without context.
Is it possible to get around this problem and come up with a statistic which would allow genuine comparisons of goalkeepers? Well, if each shot was rated based on whether it should be saved or not, or perhaps given a score of 1-10 for how difficult a save it required, then that would provide the context needed to accurately compare keepers against each other. A goalkeeper making a higher proportion of difficult saves would therefore be unarguably better than a goalkeeper letting in a higher proportion of easier saves. Without this context, the statistic is meaningless.
You might suggest that these things even themselves out over a season, but this oft-used excuse seems to be accepted as conventional wisdom by many people in football without any proof whatsoever. In this instance, a solid back four will consistently lead to teams getting frustrated and shooting from distance over the course of a season, while a porous back four will consistently allow more point blank shots and one-on-ones. The number of shots taken against each side may be relatively consistent, as why would a team shoot from distance against a porous defence when they know they can create better chances by holding on to the ball? I refuse to believe that anything evens itself out over a season until somebody proves it with statistics, as there are a huge number of counterexamples, not least of which the equivalent cliché of “when you’re down there, these things go against you”. There doesn’t seem to be much “evening out over a season” going on there. Like “narrowly wide”, clichés can be oxymoronic. Or just moronic.
In conclusion, the reason for this post was just to make people stop and think when looking at the wonderful array of statistics on offer to us football fans these days. This applies as much to shot saved percentages as pass completion percentages (how easy or difficult were those passes?) or shots on target percentages (was that a free header or was he surrounded by defenders?). Context is the key to statistical analysis.