Ranking batters by average when number of observations (innings) vary This is my first post here and I'm new to this area so please forgive me if I'm asking a naive question.
I want to rank a number of batsmen (e.g., in cricket) by their skill. I'm planning to use batting averages as a measure of skill.
But there are some batsmen who have only been to bat a few times and therefore their average is not a good estimate of performance (I.e., I don't have a very good estimate of their "true" average).
Assuming that, for each batsman, I only have scores of each time he came to bat (e.g., 89 4, 92, 45, ... ).
How could I best sort them?
Thanks in advance.
 A: "Best" is impossible to answer since you don't give a criterion you want to optimize (that is, until you define pretty precisely what 'better' is for your purpose, 'best' isn't really possible to judge).
Trying to rank things with very different numbers of (/ precision of) observations is a common problem (think online ratings for movies for example), so you might search around for solutions to quite different problems with the same issue.
I'll present one approach I've seen used:
Take the observed value as as estimate of a population quantity (the batter's true ability*) and so calculate a one-sided interval for the quantity (a lower bound), and then order by that. You could call it an "experience-adjusted average" (or more accurately, inexperience-adjusted), which downrates the average more for less experience.
* This assumes we're not dealing with a shifting target -- if we are (and you'd expect with batting in any sport that it would shift over time), then you might weight by recency as well, such as an exponentially weighted moving average. This will still work with the above suggested approach.
Whether this suits your purpose or not, I can't say, but with a reasonably chosen interval  it often doesn't perform too badly (e.g. perhaps an 75% or an 80% interval might work okay; you can try other values).
