Our pals at STATS have a unique method to evaluate glovesmen and separate the wheat from chaff. Kevin Chroust and Paul Power explain all...
Alisson’s status as the world’s most expensive goalkeeper lasted roughly three weeks until Chelsea snapped up Kepa Arrizibalaga just before the start of the season.
Each decision was based on a clear need – Chelsea to replace the Real Madrid-bound Thibaut Courtois, and Liverpool to ensure what happened in the 2018 Champions League Final wasn’t ever repeated on Jurgen Klopp’s watch.
But is it all that straightforward?
Kepa's transfer fee had everything to do with his buyout clause, but Chelsea must have felt he was at least approaching Alisson’s level to justify the cost. In hindsight, most would say that Alisson has been the better keeper, but how could we have objectively and meaningfully assessed this at the time? And how might we have considered less expensive options that could produce similar results?
This is difficult, because traditional goalkeeper measurements such as clean sheets and save percentages aren’t truly indicative of performance. A goalkeeper’s value is difficult to quantify, even with advanced metrics such as expected saves (xS), because they only measure how a player performed against the shots they faced. Keepers may have completely different types of saves to make depending on the defensive style of their team and the opponents they face.
Method, man
So rather than using metrics that might not capture these variables, we can simulate each goalkeeper for every shot, then compare who would concede the fewest goals. To do this, we must be able to accurately simulate how one keeper would manage against another keeper’s shots – or better yet, how each keeper would perform against every shot in a uniform sample. That’s exactly what we’ll do here.
Get FourFourTwo Newsletter
The best features, fun and footballing quizzes, straight to your inbox every week.
The following table shows an unsurprising duo at the top. When measured against the same shot sample of 2,754 shots from the 2018/19 season, David de Gea and Alisson are capable of saving a team more than a goal every four matches. They truly are season-altering shot-stoppers for any club:
Before diving too deep, we’ll briefly explain the method, then get back to a larger Premier League goalkeeper analysis with a first-of-its-kind shot-stopping simulation.
Here’s how it’s done.
Using tracking data and machine learning, we can create a set of features that measure keepers’ strengths and weaknesses when facing shots in different situations. For example, are they stronger to their left or right sides? How do they handle 1-vs-1 situations? Are they easily beaten at their feet?
Once we generate these features, we combine them into what we call a Player Embedding, which creates a unique data-driven goalkeeper identity. When we plot these embeddings, we’re able to pick up on the unique nuances of each one (see the published paper linked below for full details on how this is calculated).
If we combine these embeddings with a normal xS model, we significantly improve our accuracy in predicting whether a goalkeeper will save a shot on target. This allows us to move into the area of personalised player prediction. As a result, if we want to know how Goalkeeper A would cope facing Goalkeeper B’s shots – or in our case, how Alisson would handle Kepa’s shots – we can now take each player’s unique identity and swap them to simulate what we’d expect to happen.
Now back to the fun part.
NEXT: Why Kepa's not quite stacking up...
Is he a Kepa?
We probably didn’t surprise you with news that Alisson would outperform Kepa and other Premier League newcomers when measured against the same sample of shots. But we might spark a debate with some of the other findings.
As above, Kepa is a top-third keeper in the Premier League – but when compared to some of the division’s other newcomers, he’s not differentiating himself as much as his price tag would suggest. Vicente Guaita, who joined Crystal Palace from Getafe on a free transfer, is a top-five keeper when given the same shot sample opportunity as all other top-flight keepers. Bernd Leno and Rui Patricio – comparatively inexpensive options for Arsenal and Wolves – aren’t far behind.
All fascinating stuff, but what if we need to see one keeper’s strengths compared to another’s? In the following visual, we’re aggregating all shots faced by both keepers and showing which on-target areas each keeper displays strengths over the other:
Alisson, we can see, would concede seven fewer goals than Kepa in this shot sample. He’d do so because he’s a comparatively exceptional keeper facing low, centre and right-of-centre chances. Kepa, meanwhile, is better going to his left, though with far less contrast.
Guaita vs Kepa in Spain
This is information Chelsea would be receiving in hindsight, but we can also go back to the 2017/18 season and see how Kepa compared in La Liga. Guaita was also there then, so we’ll bring him in too.
Kepa conceded 43 goals in 30 matches while making 96 saves for an Athletic Bilbao side that finished fifth-bottom. Guaita conceded 26 goals in 34 matches while making 85 saves for a defensive-minded Getafe side that conceded fewer goals than Champions League holders Real Madrid and Champions League qualifiers Valencia. It seems like a transfer that Palace deserve more credit for than they’ve received.
But how do the two keepers rank using embeddings to measure against a uniform shot sample? It favours the expert’s eye test over the basic team data outlined above: Kepa was better than Guaita, but not substantially. The 24-year-old ranked seventh in the division at 0.10 xG prevented per 90, while Guaita was eighth at 0.097. For the sake of comparison with our gold standard, Alisson was second in Serie A at 0.33 xG prevented per 90, behind Samir Handanovic.
It’s even more interesting when we consider Kepa and Guaita directly against each other in 2017/18:
Kepa (now red) still displays superiority going wide left when compared to Guaita, but he’s also mimicking some of the strengths Alisson held over him (low and right of centre).
Here and now
If we zoom back out to the larger context of the 2018/19 Premier League, we’ll break this down matchday by matchday to see how these first-year EPL keepers have evolved over the season in comparison with three others:
Alisson was the best keeper in the Premier League through to Matchday 21, and David de Gea has held that title since. Kepa has been above average all season but not exceptional.
Leno had to earn his position and didn’t play particularly well when he took over from Petr Cech, but he’s steadily improved. Rui Patricio has also been above average in his first season but peaked early.
It’s no secret that world-record signings don’t necessarily mean world-class performances. But a smarter way to make data-driven recruitment decisions has arrived.
