How to rank the players

Question

I have made an app that keeps track of the games in our little floorball club. We play once a week. The teams are made random for each game. Some people show up every time and some people less frequent. We are always enough to divide two teams and play 3-4 games. Sometimes we mix up the teams during the evening.

We record the teams for each game and the name of the player for each goal scored.

This is a great basis for all sorts of statistics over time.

Currently the main result I pull out of the data is a list of all players ordered by the ratio between wins and total games for each player.

So the player who has won all his games is at the top of the list.

The problem right now is that my little brother showed up to one game which he won. Then he got injured and has been stuck at the top of the list with a 100% wins ever since. Very annoying!

The question: What would be a fair way to adjust this list, so you somehow reward the people who show up every time. Or at least punish people who does not have enough games to have a realistic score.

(Please comment if you have ideas to other interesting results, that I can pull out of my data.)

Answer 1

You should try the empirical Bayesian approach. You can see an example related to baseball here. This gist though is that you update players as you gain more observations thus giving you more reason to move from your prior. This is because having a winning percentage of 100 means something very different for 1/1 vs 100/100. In the prior case one loss will make you 50% while in the later case one loss keeps you pretty close to 100. This method will most likely move your brother to the average player.

Answer 2

Typically you would do credibility weighting of some kind to ensure that if you have small or insufficient statistics for a person their overall average is closer to the average of the overall group.

In particular you might employ an equation like that in the wikipedia article on credibility theory

$$ S = z_j \bar{X}_j + (1-z_j) \bar{X}$$ where $S$ is average win/score, $z_j$ represents how credible the group $j$ is with 0 meaning not at all and 1 meaning totally, $X_j$ is average for group $j$ and $X$ is average overall. As a first step you could take $z_j$ to be the proportion of games played.