ephex posted:

INTRODUCTION:
Expected goals is a rising buzzword in football these days, a trend definitely noticeable on our sub. Most of us have a good idea on what it tries to capture and a somewhat vague idea on how exactly they capture it.
I am a data scientist by profession and I have freelanced for websites/startups which work on developing models for these metrics. I have seen that a lot of people have started xG as a metric around football discussions, which is a good thing. Most people on here seem to understand what it tries to do. But a lot of them don't know how it tries to achieve it's objectives. I have decided to make a post on the typical strategies used to calculate these metrics. And analyze the good and the bad in using xG to analyze games/teams/players.

The Basic Definition:
xG: Expected number of goals scored by a team. It aims to measure, how many goals a team is likely to score in that match taking into account the quality of chances created.

xGA: Expected number of goals conceded by team. It aims to measure the number of goals a team is likely to concede taking into account the quality of chances conceded.

Base Strategy/Idea used by all:
Shots matter, and only the shots matter: Every shot taken by a team is assigned a specific chance of ending up in goal. Which is taken as the xG for that shot. Sum up the xG assigned for all the shots. You are done.

If you do not take a shot, you don't get any xG no matter how promising a situation it was.

NOTE: We are using shots, and not shots on target, which obviously makes sense, Just because a shot ended up being off target doesn't mean that you had no possibility to score from that chance

Standard Factors affecting the assignment
Walking it in :
The field is divided into zones and sub areas, a huge number of statistics are available for each sub area (WhoScored implement a basic version of this stat) about the number of shots taken in the area and the number of goals scored from that area.

Example: Let us say, one of the zones is the left corner of the 6 yard box. We know that a 1000 shots have been taken from that area in the past, we also know that 700 goals have been scored from that area in the past. So let us assign each shot from that area as having a 0.7 expected goal value.

Tony Pulis, anyone?
There are also certain situations in football, which have a special xG placed on them. Shots from corner kicks, shots from free kicks (from different positions), and Penalties. They also use the same principle of dividing total goals from the situation by the total shots from the situation and assign it as xG.

Example:

Assume there have been 1000 corners taken in the past. Assume that a total of 150 goals have been scored from corners, so each corner kick will have a 0.15 expected goal value (* As long as it does lead to a shot).

Same for free kicks, except free kicks are also combined with zonal data.

Penalties are easiest to calculate simply divide number of penalty goals by total number of penalties taken (In practice they just maintaining a running average than dividing sum totals). They are usually fixed to be around 0.8

Using the right parts
Weak foot shots will have a lower xG, because stats show that they have a lower chance of succeeding. (Only some models take this into account, so beware!)

Headers will also have a different xG for a given zone. (Almost all models implement this) Calculations are similar, simply find the ratio of goals to attempted header statistics.

Black Magic?
These are certain obscure adjustments that some companies will often do. The exact methods here are mainly proprietary and usually considered trade secrets, and a lot of wtf hacks are going on. But i will give a general idea of what is being done.

Direct Adjustment:

Combining chances by multiple criteria: Headers from corners, combined with zonal data.
xG by phase: 2nd headers from set pieces, penalty rebounds.
Counter Attacks:

One on ones are given special consideration.
Number of defenders in the box is also considered for adjustment. But their distance/closeness to the shooting player is not (This is important will explain later).
NOTE: A lot of companies do not perform any of these black magics and just give a raw indication of xG. But some companies truly go above and beyond and use neural networks and sequential modelling to consider multiple phases of play.

Judgement Time:
Let us take a look at what we can accurately infer from these stats and what we have to be careful about.

The good:
It is easy to see what we gain from xG (if we assume it is accurate), so we won't go into detailed examples to prove it's worth.

Identifying Strengths and Weaknesses:

A team with a very high xG over a season with a lot of goals very likely means they have done very well to create lots of quality chances (See Man City, Barcelona, Real Madrid, Bayern).
A team with a consistently high goals scored vs xG indicates excellent shooters.
A team with a high xG but lesser goals scored can be due to poor finishing, or being unlucky, or it can also mean that they are taking a lot of useless shots from outside the box.
A team with a very high xGA for set pieces indicates poor preparedness from set pieces.
A team with very high xGA but few goals conceded indicates a good Goal Keeper (Man United, Atletico Madird). and the reverse indicates a poorer goal keeper (Arsenal, Liverpool (mainly Mignolet)).
Identifying individual player qualities: Most xG data is also available for individual players, Insights into their strengths and weaknesses can also be gained using stats.

Weak foot xG compared to weak foot goals will indicate how good a player's week foot is
xG vs actual goals scored gives an indication of finishing ability and also may indicate ability to score difficult chances.
xGA vs number of goals conceded can give a good indication of a goalkeeper's ability as long as the defence is not too bad or too amazing to skew the stats.
Identifying Systems and Tactics

A team with high number of goals scored with a low xG usually indicates an effective counter attacking team. (See Leicester, Atletico Madrid)
A team with low xGA and even lower goals conceded indicates a possession dominating team with very good individual defenders.
A team with low xGA and but a relatively high goals conceded indicates a possession dominating team with relatively poor individual defenders or poor goalkeepers.
The Bad:
This will be much longer than the good, even though I think xG overall is pretty useful because it is harder to see what is bad than what is good.

Danny Welbeck vs Cristiano Ronaldo

xG does not take into account the previous record for a single player. In a given position in a given situation, Danny Welbeck has the same xG to score as Cristiano Ronaldo. Of course this will help when comparing players and we can conclude that Ronaldo is a better finisher. But if we use it to argue that Arsenal had the same chance to score as Real Madrid, then it becomes false, because it doesn't take into account that Ronaldo is the better finisher.

Shoot, shoot and shoot again?

The xG system typically has a minimum value below which it is clipped off. Say 0.1 This means that no matter how bad a shot is it will have an xG of at least 0.1.

Example: Let us say, Ander Herrera decided to take 10 shots from outside the box, as is typical none of them have a chance in hell of ever being a goal, but by definition his total xG is 1. Contrast to say Lukaku being in a 1 on 1 situation only once with a xG of 0.7. I will take the latter situation every game.

Missing Sitters vs Scoring Worldies

This also means that players who shoot a lot, tend to have lesser goals than their xG or almost equal to xG, even if they are very good finishers. And players who shoot very less and score from counters tend to have a lot more goals then their xG would suggest This can be offset when players who shoot a lot actually manage to score off long shots, since that gives a huge jump in outperforming xG.

eg: Kane has outperformed his xG in the past seasons, despite having a reputation for taking a lot of shots which indicates that he is a very good finisher and also has a very good ability to convert long shots / difficult chances. This season however Kane has an xG of 23.99 and has scored 24 goals, which indicates that he has not put away difficult chances as much as previous seasons.

Salah has an xG of 21.79 but has scored 29. And he has also consistently outperformed his xG in his past seasons. This despite him having a slight reputation for missing some easy chances. Which indicates that he shoots lesser than forwards like Kane/Ronaldo and a lot of his chances come from counter attacks and also that he has the ability to convert really difficult chances.

Note: The fact that Salah has consistently outperformed his xG and his even larger jump in outperforming his xG for Liverpool means that as long as Liverpool play a similar style Salah is very likely to keep scoring at this alarming rate. (This is absolutely horrifying for me as a United fan, do something papa flo, pls.)

Context vs Random Conclusion

The above section is a bit confusing and there was no clear conclusion to what exactly does it indicate when a player outperforms his xG. This basically means that there are various ways the xG and goals scored can go depending on player attributes.

So take into account the context and past patterns before coming to conclusions. If someone keeps performing more than their xG for multiple season, it is not luck, it means that there are somethings that the model doesn't capture.

Tiki Taka vs Counter Attack

Let us come to one of the biggest flaws in xG. As much as it tries not to, currently xG always paints counterattacking/park the bus teams in poor light.

Let us take a look at one of my favorite examples of a team controlling the game without having the ball.

Manchester United 2 - 0 Ajax (Europa League Final)

This was a masterpiece of a match to indicate what is wrong with stats in football. Especially applying a long term stat like xG to a single one off game.

Ajax had a whopping 17 shots, with a respectable xG of 1.6, Manchester United had 7 shots with an xG of 1.1.

Ajax dominated every passing stat, possession stats, touches inside the penalty area stats.

However anyone who saw the match could have told you that Ajax could have played for an entire day and still would not have come anywhere near scoring. That's how good United were defensively and tactically.

If you saw the match and use the context along with the stats, you can safely conclude that Manchester United were extremely comfortable with allowing Ajax to have the ball and attempt speculative shots from distance while closing down angles. Which is indicated by the fact that even with such a large number of shots Ajax had quite a poor xG.

Without seeing the match however, it can be easy to wrongly conclude that Ajax dominated and United got a flukey win.

Teams which sit back and absorb pressure and the counter are hard done by the xG stats. Both in xGA and xG. One of the biggest reasons for it is:

Most models don't account for the number of defenders involved while defending an attack.
Some models try to use a threshold number of defenders, below which it is considered a fast attack and thus the xG is adjusted. (This is better than the previous case, but still not good enough)
They don't take into account the relative distance of defenders and the shot taker.
This means that it doesn't take into account the defensive organization of a parked bus, where players close down and block the angles or block the shot.

This also means that it doesn't take into account the vulnerability of high line teams when faced with a good counterattack because, even though it is easier to score from 20 yards with 1 or 2 defenders ahead of you as opposed to a parked bus the xG is going to be same.

Let us take a look at some of the teams that have become famous for a good bus/jammy 1-0 wins:

Team GA xGA
Manchester United 23 35.09
Burnley 27 41.88
Atletico Madrid 14 27.40
Juventus 16 21.81
These are all huge differentials, and all of these teams have a somewhat similar style of defending, that and a good goalkeeper means that they are comfortable facing a lot of shots/inviting pressure. It is unfair to conclude that they are good simply because of a goalkeeper bailing them out, it is the system + the GK.

Note: xG stats are taken from : understat

The Future
A lot of the flaws in xG are very difficult to rectify. And companies are reluctant to reveal their exact algorithms, which is understandable.

I have switched jobs, but since i love football, I still work on some things as a hobby. I am currently working on a way to model the exact relative positions of defenders when a shot was made using stochastic geometry and take that into account while calculating xG, which if successful can drastically improve it's reliability.

If i have the time and motivation I will finish it and release it to the public with comprehensive documentation for the exact methodology used. (Don't hold your breath though, i am incredibly lazy and have a short attention span.)

The Verdict
xG is here to stay. It has it's flaws, but with context it can be a good statistic to analyze a team's style of play, It can be used as an indicator of their future performances.

But we have to be very careful in using it to explain past performances and even more careful in using it to analyze one off performances. There can be statistical anomalies, teams can genuinely perform better than expected, but it is likely that they will revert to the mean.

Use it to analyze the teams in the long term rather than lording it over someone that you deserved better.

Note: Anyone who is interested can pm me with specific questions to ask for more details regarding the statistical inference methods used to tweak the calculations, but I won't be able to pinpoint the exact values and algorithms used because they are trade secrets.

TL;DR: A tl;dr defeats the entire point of my post, but still here it goes: "xG with context good, xG without context bad".

ephex posted:

INTRODUCTION:
Expected goals is a rising buzzword in football these days, a trend definitely noticeable on our sub. Most of us have a good idea on what it tries to capture and a somewhat vague idea on how exactly they capture it.
I am a data scientist by profession and I have freelanced for websites/startups which work on developing models for these metrics. I have seen that a lot of people have started xG as a metric around football discussions, which is a good thing. Most people on here seem to understand what it tries to do. But a lot of them don't know how it tries to achieve it's objectives. I have decided to make a post on the typical strategies used to calculate these metrics. And analyze the good and the bad in using xG to analyze games/teams/players.

The Basic Definition:
xG: Expected number of goals scored by a team. It aims to measure, how many goals a team is likely to score in that match taking into account the quality of chances created.

xGA: Expected number of goals conceded by team. It aims to measure the number of goals a team is likely to concede taking into account the quality of chances conceded.

Base Strategy/Idea used by all:
Shots matter, and only the shots matter: Every shot taken by a team is assigned a specific chance of ending up in goal. Which is taken as the xG for that shot. Sum up the xG assigned for all the shots. You are done.

If you do not take a shot, you don't get any xG no matter how promising a situation it was.

NOTE: We are using shots, and not shots on target, which obviously makes sense, Just because a shot ended up being off target doesn't mean that you had no possibility to score from that chance

Standard Factors affecting the assignment
Walking it in :
The field is divided into zones and sub areas, a huge number of statistics are available for each sub area (WhoScored implement a basic version of this stat) about the number of shots taken in the area and the number of goals scored from that area.

Example: Let us say, one of the zones is the left corner of the 6 yard box. We know that a 1000 shots have been taken from that area in the past, we also know that 700 goals have been scored from that area in the past. So let us assign each shot from that area as having a 0.7 expected goal value.

Tony Pulis, anyone?
There are also certain situations in football, which have a special xG placed on them. Shots from corner kicks, shots from free kicks (from different positions), and Penalties. They also use the same principle of dividing total goals from the situation by the total shots from the situation and assign it as xG.

Example:

Assume there have been 1000 corners taken in the past. Assume that a total of 150 goals have been scored from corners, so each corner kick will have a 0.15 expected goal value (* As long as it does lead to a shot).

Same for free kicks, except free kicks are also combined with zonal data.

Penalties are easiest to calculate simply divide number of penalty goals by total number of penalties taken (In practice they just maintaining a running average than dividing sum totals). They are usually fixed to be around 0.8

Using the right parts
Weak foot shots will have a lower xG, because stats show that they have a lower chance of succeeding. (Only some models take this into account, so beware!)

Headers will also have a different xG for a given zone. (Almost all models implement this) Calculations are similar, simply find the ratio of goals to attempted header statistics.

Black Magic?
These are certain obscure adjustments that some companies will often do. The exact methods here are mainly proprietary and usually considered trade secrets, and a lot of wtf hacks are going on. But i will give a general idea of what is being done.

Direct Adjustment:

Combining chances by multiple criteria: Headers from corners, combined with zonal data.
xG by phase: 2nd headers from set pieces, penalty rebounds.
Counter Attacks:

One on ones are given special consideration.
Number of defenders in the box is also considered for adjustment. But their distance/closeness to the shooting player is not (This is important will explain later).
NOTE: A lot of companies do not perform any of these black magics and just give a raw indication of xG. But some companies truly go above and beyond and use neural networks and sequential modelling to consider multiple phases of play.

Judgement Time:
Let us take a look at what we can accurately infer from these stats and what we have to be careful about.

The good:
It is easy to see what we gain from xG (if we assume it is accurate), so we won't go into detailed examples to prove it's worth.

Identifying Strengths and Weaknesses:

A team with a very high xG over a season with a lot of goals very likely means they have done very well to create lots of quality chances (See Man City, Barcelona, Real Madrid, Bayern).
A team with a consistently high goals scored vs xG indicates excellent shooters.
A team with a high xG but lesser goals scored can be due to poor finishing, or being unlucky, or it can also mean that they are taking a lot of useless shots from outside the box.
A team with a very high xGA for set pieces indicates poor preparedness from set pieces.
A team with very high xGA but few goals conceded indicates a good Goal Keeper (Man United, Atletico Madird). and the reverse indicates a poorer goal keeper (Arsenal, Liverpool (mainly Mignolet)).
Identifying individual player qualities: Most xG data is also available for individual players, Insights into their strengths and weaknesses can also be gained using stats.

Weak foot xG compared to weak foot goals will indicate how good a player's week foot is
xG vs actual goals scored gives an indication of finishing ability and also may indicate ability to score difficult chances.
xGA vs number of goals conceded can give a good indication of a goalkeeper's ability as long as the defence is not too bad or too amazing to skew the stats.
Identifying Systems and Tactics

A team with high number of goals scored with a low xG usually indicates an effective counter attacking team. (See Leicester, Atletico Madrid)
A team with low xGA and even lower goals conceded indicates a possession dominating team with very good individual defenders.
A team with low xGA and but a relatively high goals conceded indicates a possession dominating team with relatively poor individual defenders or poor goalkeepers.
The Bad:
This will be much longer than the good, even though I think xG overall is pretty useful because it is harder to see what is bad than what is good.

Danny Welbeck vs Cristiano Ronaldo

xG does not take into account the previous record for a single player. In a given position in a given situation, Danny Welbeck has the same xG to score as Cristiano Ronaldo. Of course this will help when comparing players and we can conclude that Ronaldo is a better finisher. But if we use it to argue that Arsenal had the same chance to score as Real Madrid, then it becomes false, because it doesn't take into account that Ronaldo is the better finisher.

Shoot, shoot and shoot again?

The xG system typically has a minimum value below which it is clipped off. Say 0.1 This means that no matter how bad a shot is it will have an xG of at least 0.1.

Example: Let us say, Ander Herrera decided to take 10 shots from outside the box, as is typical none of them have a chance in hell of ever being a goal, but by definition his total xG is 1. Contrast to say Lukaku being in a 1 on 1 situation only once with a xG of 0.7. I will take the latter situation every game.

Missing Sitters vs Scoring Worldies

This also means that players who shoot a lot, tend to have lesser goals than their xG or almost equal to xG, even if they are very good finishers. And players who shoot very less and score from counters tend to have a lot more goals then their xG would suggest This can be offset when players who shoot a lot actually manage to score off long shots, since that gives a huge jump in outperforming xG.

eg: Kane has outperformed his xG in the past seasons, despite having a reputation for taking a lot of shots which indicates that he is a very good finisher and also has a very good ability to convert long shots / difficult chances. This season however Kane has an xG of 23.99 and has scored 24 goals, which indicates that he has not put away difficult chances as much as previous seasons.

Salah has an xG of 21.79 but has scored 29. And he has also consistently outperformed his xG in his past seasons. This despite him having a slight reputation for missing some easy chances. Which indicates that he shoots lesser than forwards like Kane/Ronaldo and a lot of his chances come from counter attacks and also that he has the ability to convert really difficult chances.

Note: The fact that Salah has consistently outperformed his xG and his even larger jump in outperforming his xG for Liverpool means that as long as Liverpool play a similar style Salah is very likely to keep scoring at this alarming rate. (This is absolutely horrifying for me as a United fan, do something papa flo, pls.)

Context vs Random Conclusion

The above section is a bit confusing and there was no clear conclusion to what exactly does it indicate when a player outperforms his xG. This basically means that there are various ways the xG and goals scored can go depending on player attributes.

So take into account the context and past patterns before coming to conclusions. If someone keeps performing more than their xG for multiple season, it is not luck, it means that there are somethings that the model doesn't capture.

Tiki Taka vs Counter Attack

Let us come to one of the biggest flaws in xG. As much as it tries not to, currently xG always paints counterattacking/park the bus teams in poor light.

Let us take a look at one of my favorite examples of a team controlling the game without having the ball.

Manchester United 2 - 0 Ajax (Europa League Final)

This was a masterpiece of a match to indicate what is wrong with stats in football. Especially applying a long term stat like xG to a single one off game.

Ajax had a whopping 17 shots, with a respectable xG of 1.6, Manchester United had 7 shots with an xG of 1.1.

Ajax dominated every passing stat, possession stats, touches inside the penalty area stats.

However anyone who saw the match could have told you that Ajax could have played for an entire day and still would not have come anywhere near scoring. That's how good United were defensively and tactically.

If you saw the match and use the context along with the stats, you can safely conclude that Manchester United were extremely comfortable with allowing Ajax to have the ball and attempt speculative shots from distance while closing down angles. Which is indicated by the fact that even with such a large number of shots Ajax had quite a poor xG.

Without seeing the match however, it can be easy to wrongly conclude that Ajax dominated and United got a flukey win.

Teams which sit back and absorb pressure and the counter are hard done by the xG stats. Both in xGA and xG. One of the biggest reasons for it is:

Most models don't account for the number of defenders involved while defending an attack.
Some models try to use a threshold number of defenders, below which it is considered a fast attack and thus the xG is adjusted. (This is better than the previous case, but still not good enough)
They don't take into account the relative distance of defenders and the shot taker.
This means that it doesn't take into account the defensive organization of a parked bus, where players close down and block the angles or block the shot.

This also means that it doesn't take into account the vulnerability of high line teams when faced with a good counterattack because, even though it is easier to score from 20 yards with 1 or 2 defenders ahead of you as opposed to a parked bus the xG is going to be same.

Let us take a look at some of the teams that have become famous for a good bus/jammy 1-0 wins:

Team GA xGA
Manchester United 23 35.09
Burnley 27 41.88
Atletico Madrid 14 27.40
Juventus 16 21.81
These are all huge differentials, and all of these teams have a somewhat similar style of defending, that and a good goalkeeper means that they are comfortable facing a lot of shots/inviting pressure. It is unfair to conclude that they are good simply because of a goalkeeper bailing them out, it is the system + the GK.

Note: xG stats are taken from : understat

The Future
A lot of the flaws in xG are very difficult to rectify. And companies are reluctant to reveal their exact algorithms, which is understandable.

I have switched jobs, but since i love football, I still work on some things as a hobby. I am currently working on a way to model the exact relative positions of defenders when a shot was made using stochastic geometry and take that into account while calculating xG, which if successful can drastically improve it's reliability.

If i have the time and motivation I will finish it and release it to the public with comprehensive documentation for the exact methodology used. (Don't hold your breath though, i am incredibly lazy and have a short attention span.)

The Verdict
xG is here to stay. It has it's flaws, but with context it can be a good statistic to analyze a team's style of play, It can be used as an indicator of their future performances.

But we have to be very careful in using it to explain past performances and even more careful in using it to analyze one off performances. There can be statistical anomalies, teams can genuinely perform better than expected, but it is likely that they will revert to the mean.

Use it to analyze the teams in the long term rather than lording it over someone that you deserved better.

Note: Anyone who is interested can pm me with specific questions to ask for more details regarding the statistical inference methods used to tweak the calculations, but I won't be able to pinpoint the exact values and algorithms used because they are trade secrets.

TL;DR: A tl;dr defeats the entire point of my post, but still here it goes: "xG with context good, xG without context bad".

ephex posted:

TL;DR: A tl;dr defeats the entire point of my post, but still here it goes: "xG with context good, xG without context bad".

TheBigAristotle posted:

big crush on Chad OMG posted:

I stopped reading when I saw xG

ephex posted:

INTRODUCTION:
Expected goals is a rising buzzword in football these days, a trend definitely noticeable on our sub. Most of us have a good idea on what it tries to capture and a somewhat vague idea on how exactly they capture it.
I am a data scientist by profession and I have freelanced for websites/startups which work on developing models for these metrics. I have seen that a lot of people have started xG as a metric around football discussions, which is a good thing. Most people on here seem to understand what it tries to do. But a lot of them don't know how it tries to achieve it's objectives. I have decided to make a post on the typical strategies used to calculate these metrics. And analyze the good and the bad in using xG to analyze games/teams/players.

The Basic Definition:
xG: Expected number of goals scored by a team. It aims to measure, how many goals a team is likely to score in that match taking into account the quality of chances created.

xGA: Expected number of goals conceded by team. It aims to measure the number of goals a team is likely to concede taking into account the quality of chances conceded.

Base Strategy/Idea used by all:
Shots matter, and only the shots matter: Every shot taken by a team is assigned a specific chance of ending up in goal. Which is taken as the xG for that shot. Sum up the xG assigned for all the shots. You are done.

If you do not take a shot, you don't get any xG no matter how promising a situation it was.

NOTE: We are using shots, and not shots on target, which obviously makes sense, Just because a shot ended up being off target doesn't mean that you had no possibility to score from that chance

Standard Factors affecting the assignment
Walking it in :
The field is divided into zones and sub areas, a huge number of statistics are available for each sub area (WhoScored implement a basic version of this stat) about the number of shots taken in the area and the number of goals scored from that area.

Example: Let us say, one of the zones is the left corner of the 6 yard box. We know that a 1000 shots have been taken from that area in the past, we also know that 700 goals have been scored from that area in the past. So let us assign each shot from that area as having a 0.7 expected goal value.

Tony Pulis, anyone?
There are also certain situations in football, which have a special xG placed on them. Shots from corner kicks, shots from free kicks (from different positions), and Penalties. They also use the same principle of dividing total goals from the situation by the total shots from the situation and assign it as xG.

Example:

Assume there have been 1000 corners taken in the past. Assume that a total of 150 goals have been scored from corners, so each corner kick will have a 0.15 expected goal value (* As long as it does lead to a shot).

Same for free kicks, except free kicks are also combined with zonal data.

Penalties are easiest to calculate simply divide number of penalty goals by total number of penalties taken (In practice they just maintaining a running average than dividing sum totals). They are usually fixed to be around 0.8

Using the right parts
Weak foot shots will have a lower xG, because stats show that they have a lower chance of succeeding. (Only some models take this into account, so beware!)

Headers will also have a different xG for a given zone. (Almost all models implement this) Calculations are similar, simply find the ratio of goals to attempted header statistics.

Black Magic?
These are certain obscure adjustments that some companies will often do. The exact methods here are mainly proprietary and usually considered trade secrets, and a lot of wtf hacks are going on. But i will give a general idea of what is being done.

Direct Adjustment:

Combining chances by multiple criteria: Headers from corners, combined with zonal data.
xG by phase: 2nd headers from set pieces, penalty rebounds.
Counter Attacks:

One on ones are given special consideration.
Number of defenders in the box is also considered for adjustment. But their distance/closeness to the shooting player is not (This is important will explain later).
NOTE: A lot of companies do not perform any of these black magics and just give a raw indication of xG. But some companies truly go above and beyond and use neural networks and sequential modelling to consider multiple phases of play.

Judgement Time:
Let us take a look at what we can accurately infer from these stats and what we have to be careful about.

The good:
It is easy to see what we gain from xG (if we assume it is accurate), so we won't go into detailed examples to prove it's worth.

Identifying Strengths and Weaknesses:

A team with a very high xG over a season with a lot of goals very likely means they have done very well to create lots of quality chances (See Man City, Barcelona, Real Madrid, Bayern).
A team with a consistently high goals scored vs xG indicates excellent shooters.
A team with a high xG but lesser goals scored can be due to poor finishing, or being unlucky, or it can also mean that they are taking a lot of useless shots from outside the box.
A team with a very high xGA for set pieces indicates poor preparedness from set pieces.
A team with very high xGA but few goals conceded indicates a good Goal Keeper (Man United, Atletico Madird). and the reverse indicates a poorer goal keeper (Arsenal, Liverpool (mainly Mignolet)).
Identifying individual player qualities: Most xG data is also available for individual players, Insights into their strengths and weaknesses can also be gained using stats.

Weak foot xG compared to weak foot goals will indicate how good a player's week foot is
xG vs actual goals scored gives an indication of finishing ability and also may indicate ability to score difficult chances.
xGA vs number of goals conceded can give a good indication of a goalkeeper's ability as long as the defence is not too bad or too amazing to skew the stats.
Identifying Systems and Tactics

A team with high number of goals scored with a low xG usually indicates an effective counter attacking team. (See Leicester, Atletico Madrid)
A team with low xGA and even lower goals conceded indicates a possession dominating team with very good individual defenders.
A team with low xGA and but a relatively high goals conceded indicates a possession dominating team with relatively poor individual defenders or poor goalkeepers.
The Bad:
This will be much longer than the good, even though I think xG overall is pretty useful because it is harder to see what is bad than what is good.

Danny Welbeck vs Cristiano Ronaldo

xG does not take into account the previous record for a single player. In a given position in a given situation, Danny Welbeck has the same xG to score as Cristiano Ronaldo. Of course this will help when comparing players and we can conclude that Ronaldo is a better finisher. But if we use it to argue that Arsenal had the same chance to score as Real Madrid, then it becomes false, because it doesn't take into account that Ronaldo is the better finisher.

Shoot, shoot and shoot again?

The xG system typically has a minimum value below which it is clipped off. Say 0.1 This means that no matter how bad a shot is it will have an xG of at least 0.1.

Example: Let us say, Ander Herrera decided to take 10 shots from outside the box, as is typical none of them have a chance in hell of ever being a goal, but by definition his total xG is 1. Contrast to say Lukaku being in a 1 on 1 situation only once with a xG of 0.7. I will take the latter situation every game.

Missing Sitters vs Scoring Worldies

This also means that players who shoot a lot, tend to have lesser goals than their xG or almost equal to xG, even if they are very good finishers. And players who shoot very less and score from counters tend to have a lot more goals then their xG would suggest This can be offset when players who shoot a lot actually manage to score off long shots, since that gives a huge jump in outperforming xG.

eg: Kane has outperformed his xG in the past seasons, despite having a reputation for taking a lot of shots which indicates that he is a very good finisher and also has a very good ability to convert long shots / difficult chances. This season however Kane has an xG of 23.99 and has scored 24 goals, which indicates that he has not put away difficult chances as much as previous seasons.

Salah has an xG of 21.79 but has scored 29. And he has also consistently outperformed his xG in his past seasons. This despite him having a slight reputation for missing some easy chances. Which indicates that he shoots lesser than forwards like Kane/Ronaldo and a lot of his chances come from counter attacks and also that he has the ability to convert really difficult chances.

Note: The fact that Salah has consistently outperformed his xG and his even larger jump in outperforming his xG for Liverpool means that as long as Liverpool play a similar style Salah is very likely to keep scoring at this alarming rate. (This is absolutely horrifying for me as a United fan, do something papa flo, pls.)

Context vs Random Conclusion

The above section is a bit confusing and there was no clear conclusion to what exactly does it indicate when a player outperforms his xG. This basically means that there are various ways the xG and goals scored can go depending on player attributes.

So take into account the context and past patterns before coming to conclusions. If someone keeps performing more than their xG for multiple season, it is not luck, it means that there are somethings that the model doesn't capture.

Tiki Taka vs Counter Attack

Let us come to one of the biggest flaws in xG. As much as it tries not to, currently xG always paints counterattacking/park the bus teams in poor light.

Let us take a look at one of my favorite examples of a team controlling the game without having the ball.

Manchester United 2 - 0 Ajax (Europa League Final)

This was a masterpiece of a match to indicate what is wrong with stats in football. Especially applying a long term stat like xG to a single one off game.

Ajax had a whopping 17 shots, with a respectable xG of 1.6, Manchester United had 7 shots with an xG of 1.1.

Ajax dominated every passing stat, possession stats, touches inside the penalty area stats.

However anyone who saw the match could have told you that Ajax could have played for an entire day and still would not have come anywhere near scoring. That's how good United were defensively and tactically.

If you saw the match and use the context along with the stats, you can safely conclude that Manchester United were extremely comfortable with allowing Ajax to have the ball and attempt speculative shots from distance while closing down angles. Which is indicated by the fact that even with such a large number of shots Ajax had quite a poor xG.

Without seeing the match however, it can be easy to wrongly conclude that Ajax dominated and United got a flukey win.

Teams which sit back and absorb pressure and the counter are hard done by the xG stats. Both in xGA and xG. One of the biggest reasons for it is:

Most models don't account for the number of defenders involved while defending an attack.
Some models try to use a threshold number of defenders, below which it is considered a fast attack and thus the xG is adjusted. (This is better than the previous case, but still not good enough)
They don't take into account the relative distance of defenders and the shot taker.
This means that it doesn't take into account the defensive organization of a parked bus, where players close down and block the angles or block the shot.

This also means that it doesn't take into account the vulnerability of high line teams when faced with a good counterattack because, even though it is easier to score from 20 yards with 1 or 2 defenders ahead of you as opposed to a parked bus the xG is going to be same.

Let us take a look at some of the teams that have become famous for a good bus/jammy 1-0 wins:

Team GA xGA
Manchester United 23 35.09
Burnley 27 41.88
Atletico Madrid 14 27.40
Juventus 16 21.81
These are all huge differentials, and all of these teams have a somewhat similar style of defending, that and a good goalkeeper means that they are comfortable facing a lot of shots/inviting pressure. It is unfair to conclude that they are good simply because of a goalkeeper bailing them out, it is the system + the GK.

Note: xG stats are taken from : understat

The Future
A lot of the flaws in xG are very difficult to rectify. And companies are reluctant to reveal their exact algorithms, which is understandable.

I have switched jobs, but since i love football, I still work on some things as a hobby. I am currently working on a way to model the exact relative positions of defenders when a shot was made using stochastic geometry and take that into account while calculating xG, which if successful can drastically improve it's reliability.

If i have the time and motivation I will finish it and release it to the public with comprehensive documentation for the exact methodology used. (Don't hold your breath though, i am incredibly lazy and have a short attention span.)

The Verdict
xG is here to stay. It has it's flaws, but with context it can be a good statistic to analyze a team's style of play, It can be used as an indicator of their future performances.

But we have to be very careful in using it to explain past performances and even more careful in using it to analyze one off performances. There can be statistical anomalies, teams can genuinely perform better than expected, but it is likely that they will revert to the mean.

Use it to analyze the teams in the long term rather than lording it over someone that you deserved better.

Note: Anyone who is interested can pm me with specific questions to ask for more details regarding the statistical inference methods used to tweak the calculations, but I won't be able to pinpoint the exact values and algorithms used because they are trade secrets.

TL;DR: A tl;dr defeats the entire point of my post, but still here it goes: "xG with context good, xG without context bad".

TheBigAristotle posted:

c0burn posted:

https://www.theguardian.com/football/blog/2018/apr/10/manchester-city-pep-guardiola-fury-champions-league

This was Pep-ball turned up to 11, Pep unloading the entire clip in his splurge gun right from the off, that thrilling dream of defensive control by all-out attack.

greazeball posted:

more like nosferatu

c0burn posted:

https://www.reddit.com/r/football/comments/8d2d4h/_/

TheBigAristotle posted:

pik_d posted:

https://twitter.com/City_Watch/status/986968342396395523

pik_d posted:

https://twitter.com/City_Watch/status/986968342396395523

pik_d posted:

https://twitter.com/City_Watch/status/986968342396395523

Troy Queef posted:

a: it's good art

b: it's actually referencing something fairly famous

kingturnip posted:

I was in my local newsagent this evening and while I was waiting to pay, glanced at the TV which was showing highlights of Turkish football.
I've had a quick google, but nothing's coming up in English - does anyone know why Gaziantepspor have only 1 point? From what I can tell, they've only been incredibly poo poo - as opposed to setting new standards of poo poo - with 2 wins and 3 draws all season. Did they also get a points reduction?