Technical Details – Skill Rating

Possibly the most challenging concept in the tournament to grasp is the Skill Rating. The principle itself is very simple – each player receives a rating to reflect how 'good' they are, compared to the other players.

What does confuse people is that this rating isn't dependant on the number of games won or lost. This may sound strange at first, after all we expect good players win more games than poor ones. Of course the flaw with this logic is that it depends on who the opponents were.

This necessitated knock-out tournaments to implement a seeding system to delay the meeting of top players until the later rounds. Unfortunately this results in mismatches in the earlier rounds with weaker players exiting after their first match.

Intuitively it makes more sense that a narrow loss to a good player is more of an achivement than a narrow win over a novice. So the challenge was to create a system where people weren't disadvantaged by how good their opponent was – or with doubles, how bad their partner. The approach which resulted from this philosophy was for every player to have a relative rating. Beating an opponent with a higher rating increases a player's own rating while lowering the rating of the opponent. In fact, even doing well against a higher rated opponent will lift a player's own rating.

Perhaps the greatest benefit of this approach is that it allows games to be organised in any manner desired without creating a bias in the resulting ratings. And as such, matches can be played with continually varying combinations of players of any desired level to give competitive and enjoyable games. As an individual's rating increases, they begin to be organised to play with – and against – better players.

Specifically:

Each player has a skill rating reflecting their playing ability. Every new player – being an unknown quantity – is allocated a starting rating of 10 points. This initial rating quickly adapts to the results of their first matches.

When two players combine to form a team, their individual skill ratings are combined to create a team rating. The simplest approaches would be to use the total of the individual ratings, or perhaps to average them.
When two teams compete, the probability of a particular team winning any point is proportional to their respective team ratings. For example, if team A has a rating of 12 and team B has a rating of 8, we would expect team A to win 60% of the points played. Therefore in a match of 40 points, the expected result would be 24-16 to team A.
If a game result differs from the theoretically predicted score, the ratings of all the players involved are adjusted slightly in the direction suggested by the recorded score. This way, the revised predicted score will be a step closer to the achieved score. Future predictions should be more accurate as well.
The current system adjusts the rating of newer players more than those of established players. This reflects the system having less certainty in the true ability of players with fewer games, so deviatons in game score from the predicted value are more likely due to inaccuracy in these players' ratings.

The primary challenge involved in implementing such a system is deriving a formula which realistically combines two individual ratings in to a team rating. Real life observations suggest that two players of equal ability are better than one player on his or her own.

Additionally, two good players benefit more from combining than two novice players. However, players of significantly differing abilities will have an effectiveness somewhere between the individual abilities. What's more, the opponents are likely to target the weaker player creating a bias in the true result.

A secondary challenge is how much to adjust a player rating after recording a result which differs from the predicted score. If the adjustments are too coarse, the ratings will fluctuate continually. If they are too fine, it will require too many games to reach an accurate value.

Players who have less history have a more volatile rating. As a rule of thumb it requires 15-20 recorded results before a player's ratings can be considered reliable. In fact, a player won't even have their skill rating displayed in the results until they have played at least 5 games.

While the current system attempts to strike an acceptable balance of all factors, all these considerations are constantly under review and the search for perfection is never at an end.