Hey guys, there have been quite a few threads on the SC scoring system, with opinions ranging from “it’s totally rigged so that player X always scores better than player Y” to “it’s perfect in all respects”. I thought it might be worth creating a FAQ outlining what we do know about the system. I’ll post the first draft here. If people could reply telling me where I’m wrong or unclear, that’d be great. Even if you’re not sure, if the explanation makes it seem that the system is dodgy, point out why – there’s a good chance I’ve either misunderstood it or aren’t explaining it properly.
So here goes.
So, what are these SC points?
It has been known for a long time that the two most commonly collected stats, goals and possessions, don’t tell the whole story of how well a player has performed. It’s easy to think of low possession games that have been decisive, as well as high possession ones that didn’t do much. That said, these are better than nothing. If you didn’t know anything else about two players’ performances except that one had 8 possessions and another 25, you’d always go with the 25, and you’d usually be right.
A few years ago, Champion Data and Swinburne Uni’s maths department tried to see how important various factors were in deciding who would win a game. They collected all the stats they could think of between (can anyone help me with the years?) and did an analysis to see which events correlated with teams winning games. They came up with the values you often read – 8 points for a mark from an opposition kick, 5 points for a long kick resulting in an uncontested possession to a team-mate, etc… By adding up the points for each player you could get a new stat, SC points, that is a much better indicator of how much a player did the things that make a team likely to win games. As this is proprietary knowledge that they sell, they don’t release the whole formula.
Of course, if wasn’t perfect. There were still players that seemed to get scores that didn’t correspond with our intuitive notion of how ‘good’ they were. A classic example is a negating defender, who doesn’t do much themselves, but also doesn’t let their opponent do anything either. If someone keeps Franklin to zero goals, zero disposals, but only gets a few tackles and spoils, they won’t score well – but most would say that they had a big influence on the game.
This particular flaw in the system still exists, but there are others that have been corrected as Champion Data tries to come up with better ways to measure a player's impact on the game. Now, when and where an event occurs has an effect on the score. We know that clangers in the defensive 50 get a higher weighting than ones on the wing. It’s probably safe to assume that everything gets a higher weighting if it occurs in the corridor. As well as this, everything gets weighted according to a combination of the margin and the time left in the game. The aim of this is to reward players who perform when the game is in the balance, as opposed to those who only get kicks when the result is already decided.
Isn’t it all very subjective?
Well, yes and no.
As far as the stat collecting goes, they have strict definitions for each stat. As well as this there are multiple people taking the statistics, so that errors are more likely to get picked up. It’s impossible to completely eliminate the grey areas, but this stage is about as objective as it can be. That said, the definitions themselves are not perfect. It is easy enough to think of situations where although a possession may be classed as effective (and this is objective in the sense that it clearly fits the definition) it actually hurt the team.
Once the stats are collected, everything else is automated. There is no subjective judging of momentum, context, or whatever. That said, the design of the system itself can be considered subjective in the sense that it has changed in the past and will change in the future as people come up with better ways of measuring players’ effectiveness.
Why do the scores always add up to 3300?
At the end of the game, everyone’s score is multiplied by a constant to make the total equal 3300, this is called normalisation. As an example, if there were a 100 goal match and the total came to 6600, each player’s score would be halved. They don’t look at the total near the end and start assigning clangers if it’s going to be too high. The 3300 was chosen as it was the average score from the period before they started doing this. A different number could be chosen, however, and the system would still work just as well. With 3300, the adjustment is usually quite small.
The reason for doing this is that otherwise high possession games would get players much more points than low possession games. This goes against the aim of SC points, which is to say how important a player was to their team’s chances of winning– it is clear that a seven goal performance on a wet and windy day where the scores were 10.14 to 8.9 is much more important to your team’s success than that same game at the dome in a twenty goal a side shootout. In the same way, if Adam Goodes were to play in the U/14s he would get scores of 800, as he would be clearly the most important player, this doesn’t mean he suddenly became more skilled when playing there though.
In summary, the normalization rates each player relative to the performances of the other players on the ground.
Does this penalise SC teams that have lots of players from the same club?
Yes, but not much. There are only 3300 points to dole out each match, so one player going well means that there are less available for everyone else, but this is not a large effect. Imagine a perfectly even match, where every player scored 75 points. If one player got an extra 44, this would be normalised by taking one point from every player, so there would be 43 guys with 74 points and one guy with 118. In an actual game, however, one player going well tends to draw others into the match - an effective short kick means that a teammate now has the ball and can start to rack up some points of their own.
What’s up with the situational weightings?
Before being normalized, each score is adjusted depending when in the game it occurs. The adjustment is derived by feeding the time remaining in the game and the margin into a proprietary formula. It is not based on someone subjectively saying “That was an important goal”. The aim is to adjust the weighting depending on how likely the event is to change the result of the match. We can make some assumptions about this secret formula:
-[FONT="] [/FONT]Firstly, doing things when the scores are close counts for more. This is fairly self-explanatory.
-[FONT="] [/FONT]If the scores are close, then events count for more if there isn’t much time left in the match. Again, this makes sense, you have a better chance of winning being 6 points up with one minute to go than you do being 6 points up one minute into the match.
-[FONT="] [/FONT]If it isn’t close, then doing things near the start of the game count for more. If the margin is six goals with two minutes left, then nothing you do is going to influence the result. But if it’s six goals in the second quarter, then a comeback is still possible.
There are also positional adjustments. Again, these are not publicly available, but it’s fair to assume that possessions in the corridor count for more, contested marks count for more in front of goal, etc…
Why do strange things happen when I compare the half-time scores to the full-time ones?
The half time scores are based on the assumption that there will be just as many points allocated for the second half of the game as the first. This is unlikely to be the case due to the situational weightings discussed above.
If we take the Geelong/ North game last weekend as an example you can see why. Geelong were up by 57 points at half time. At full time the result was the Cats by 70. Given that Geelong were up by so much, a very low weighting would be assigned to events in the second half, as they were very unlikely to change the result. Because of this, a player who had a big first half and then did nothing would get a higher score than one who did the reverse, the aim being to reward players who perform when the game is there to be won. In fact, the first player would have their final score be higher than their half time one even if they didn’t touch the ball.
The reverse can also happen. In a game where the margin always stays close, everything gets more and more valuable as the game goes on, as it is more likely to be game deciding. This can mean that a player who goes very well in the first half and then does little can actually have their points go down, as less than half the points end up allocated to the first half of the game.
What about taggers and defenders, they never get the ratings they deserve. Also, what about the things that SC points don’t take into account? Why are ‘professional’ free kicks penalized?
The SC points system is far from perfect. There are many things that it doesn’t take into account and it doesn’t always give players the ratings we think they deserve. It is also better than any other single number rating we have. Champion Data make their money from statistical analysis of the game, it is in their interest to come up with the best system they can, it’s just that no-one has come up with a good way to measure the influence a defender has on the game, or when it's worth it to give a free away, or when a courageous act inspires teammates, etc...
So here goes.
So, what are these SC points?
It has been known for a long time that the two most commonly collected stats, goals and possessions, don’t tell the whole story of how well a player has performed. It’s easy to think of low possession games that have been decisive, as well as high possession ones that didn’t do much. That said, these are better than nothing. If you didn’t know anything else about two players’ performances except that one had 8 possessions and another 25, you’d always go with the 25, and you’d usually be right.
A few years ago, Champion Data and Swinburne Uni’s maths department tried to see how important various factors were in deciding who would win a game. They collected all the stats they could think of between (can anyone help me with the years?) and did an analysis to see which events correlated with teams winning games. They came up with the values you often read – 8 points for a mark from an opposition kick, 5 points for a long kick resulting in an uncontested possession to a team-mate, etc… By adding up the points for each player you could get a new stat, SC points, that is a much better indicator of how much a player did the things that make a team likely to win games. As this is proprietary knowledge that they sell, they don’t release the whole formula.
Of course, if wasn’t perfect. There were still players that seemed to get scores that didn’t correspond with our intuitive notion of how ‘good’ they were. A classic example is a negating defender, who doesn’t do much themselves, but also doesn’t let their opponent do anything either. If someone keeps Franklin to zero goals, zero disposals, but only gets a few tackles and spoils, they won’t score well – but most would say that they had a big influence on the game.
This particular flaw in the system still exists, but there are others that have been corrected as Champion Data tries to come up with better ways to measure a player's impact on the game. Now, when and where an event occurs has an effect on the score. We know that clangers in the defensive 50 get a higher weighting than ones on the wing. It’s probably safe to assume that everything gets a higher weighting if it occurs in the corridor. As well as this, everything gets weighted according to a combination of the margin and the time left in the game. The aim of this is to reward players who perform when the game is in the balance, as opposed to those who only get kicks when the result is already decided.
Isn’t it all very subjective?
Well, yes and no.
As far as the stat collecting goes, they have strict definitions for each stat. As well as this there are multiple people taking the statistics, so that errors are more likely to get picked up. It’s impossible to completely eliminate the grey areas, but this stage is about as objective as it can be. That said, the definitions themselves are not perfect. It is easy enough to think of situations where although a possession may be classed as effective (and this is objective in the sense that it clearly fits the definition) it actually hurt the team.
Once the stats are collected, everything else is automated. There is no subjective judging of momentum, context, or whatever. That said, the design of the system itself can be considered subjective in the sense that it has changed in the past and will change in the future as people come up with better ways of measuring players’ effectiveness.
Why do the scores always add up to 3300?
At the end of the game, everyone’s score is multiplied by a constant to make the total equal 3300, this is called normalisation. As an example, if there were a 100 goal match and the total came to 6600, each player’s score would be halved. They don’t look at the total near the end and start assigning clangers if it’s going to be too high. The 3300 was chosen as it was the average score from the period before they started doing this. A different number could be chosen, however, and the system would still work just as well. With 3300, the adjustment is usually quite small.
The reason for doing this is that otherwise high possession games would get players much more points than low possession games. This goes against the aim of SC points, which is to say how important a player was to their team’s chances of winning– it is clear that a seven goal performance on a wet and windy day where the scores were 10.14 to 8.9 is much more important to your team’s success than that same game at the dome in a twenty goal a side shootout. In the same way, if Adam Goodes were to play in the U/14s he would get scores of 800, as he would be clearly the most important player, this doesn’t mean he suddenly became more skilled when playing there though.
In summary, the normalization rates each player relative to the performances of the other players on the ground.
Does this penalise SC teams that have lots of players from the same club?
Yes, but not much. There are only 3300 points to dole out each match, so one player going well means that there are less available for everyone else, but this is not a large effect. Imagine a perfectly even match, where every player scored 75 points. If one player got an extra 44, this would be normalised by taking one point from every player, so there would be 43 guys with 74 points and one guy with 118. In an actual game, however, one player going well tends to draw others into the match - an effective short kick means that a teammate now has the ball and can start to rack up some points of their own.
What’s up with the situational weightings?
Before being normalized, each score is adjusted depending when in the game it occurs. The adjustment is derived by feeding the time remaining in the game and the margin into a proprietary formula. It is not based on someone subjectively saying “That was an important goal”. The aim is to adjust the weighting depending on how likely the event is to change the result of the match. We can make some assumptions about this secret formula:
-[FONT="] [/FONT]Firstly, doing things when the scores are close counts for more. This is fairly self-explanatory.
-[FONT="] [/FONT]If the scores are close, then events count for more if there isn’t much time left in the match. Again, this makes sense, you have a better chance of winning being 6 points up with one minute to go than you do being 6 points up one minute into the match.
-[FONT="] [/FONT]If it isn’t close, then doing things near the start of the game count for more. If the margin is six goals with two minutes left, then nothing you do is going to influence the result. But if it’s six goals in the second quarter, then a comeback is still possible.
There are also positional adjustments. Again, these are not publicly available, but it’s fair to assume that possessions in the corridor count for more, contested marks count for more in front of goal, etc…
Why do strange things happen when I compare the half-time scores to the full-time ones?
The half time scores are based on the assumption that there will be just as many points allocated for the second half of the game as the first. This is unlikely to be the case due to the situational weightings discussed above.
If we take the Geelong/ North game last weekend as an example you can see why. Geelong were up by 57 points at half time. At full time the result was the Cats by 70. Given that Geelong were up by so much, a very low weighting would be assigned to events in the second half, as they were very unlikely to change the result. Because of this, a player who had a big first half and then did nothing would get a higher score than one who did the reverse, the aim being to reward players who perform when the game is there to be won. In fact, the first player would have their final score be higher than their half time one even if they didn’t touch the ball.
The reverse can also happen. In a game where the margin always stays close, everything gets more and more valuable as the game goes on, as it is more likely to be game deciding. This can mean that a player who goes very well in the first half and then does little can actually have their points go down, as less than half the points end up allocated to the first half of the game.
What about taggers and defenders, they never get the ratings they deserve. Also, what about the things that SC points don’t take into account? Why are ‘professional’ free kicks penalized?
The SC points system is far from perfect. There are many things that it doesn’t take into account and it doesn’t always give players the ratings we think they deserve. It is also better than any other single number rating we have. Champion Data make their money from statistical analysis of the game, it is in their interest to come up with the best system they can, it’s just that no-one has come up with a good way to measure the influence a defender has on the game, or when it's worth it to give a free away, or when a courageous act inspires teammates, etc...
