Ranking based on lower confidence interval

Question

I have a database of bridge scores from a local bridge club that effectively contains for this question, three fields: name, date and score. The score is a percentage, normally ranging from say 30% to 70%. For fun I would like to rank the players' ability.

One way would be to simply compare their means. But this doesn't take into account how often one plays (someone who consistently scores say 58% should perhaps be better ranked than someone who plays once with 60%). So I had the idea to rank by the lower bound of the 95% confidence bound calculated from their scores: lb = mean-t*s/sqrt(n).

Now I have another problem: it can be the case that a player suddenly scores a big result and by consequence lowers their ranking score. For example: after {55,56,56,57,58} the lb value is 54.98 then one tournament later with a nice 70% this becomes {55,56,56,57,58,70} with an lb value of 52.7. This non-monotonicity seems counter intuitive.

So my question: is there a better way to create such a ranking that is monotone (i.e. when a new score is better than the current mean, then the ranking doesn't decrease) and takes into account how often a player plays.

While writing this question I realised another factor that could be taken into account: the order of the inputs. The six results as presented (assuming chronological order) imply that this player is improving. Whereas if the results were really ordered {70,58,57,56,56,55} then perhaps we should conclude that the players' powers are weakening.

Answer 1

Bridge is special! In tournaments multiple pairs play the same hands, so any rating should not be based only on total score, but on how players/pairs does compared to other players/pairs of the same cards! Could also depend on type of tournament. You must also specify if you want ranking of individual players or pairs. Rating of individual players will need that individuals play in different pairs.

See for instance https://bridgewinners.com/article/view/an-elo-rating-system-for-bridge/ or https://www.bridgeworld.com/indexphp.php?page=/pages/readingroom/esoterica/bridgeratingsystem.html (lot more by googling).

Answer 2

I had a similar problem. R code demonstration.

t.test(c(1.1,2),conf.level = .9)$co
t.test(c(2.1,2),conf.level = .9)$co
t.test(c(3.1,2),conf.level = .9)$co

It is caused by the standard deviation shrinking. I offset the problem by using 75th percentile of standard deviations of all items linearly combined with actual sd. However my goals were a bit different than OP. SD suddenly increasing was not a problem only SD drastically decreasing by chance and low N. Maybe always keeping mean SD of all individuals could do what OP wants? Still my solution seems so hackey. Maybe somewhere exists a much better formulation of this problem with more formal solutions.

ttm <- function(x,psd,cl=.95,u_l="u",dsda=10){
  if(cl>=1 | cl<=0) warning("confidence level out of bounds")
  if(!is.vector(x)) warning("x not a vector")
  n <- length(x)
  if( n < 1 ) warning("x has length 0")
  if(n>1){
    nsd <- sd(x) * min( n / dsda , 1) + psd * max((1 - n / dsda) , 0)
  } else {
    nsd <- psd
  }
  ttt <- qt((1-(1-cl)/2),df = max((n-1),1))
  entrvl <- ttt * nsd / sqrt(n)
  if(u_l=="u"){  return( mean(x) + entrvl) }
  if(u_l=="l"){  return( mean(x) - entrvl) }
}

Answer 3

The considerations by Kjetil make this a complicated problem and therefore a complete answer is difficult. There is a point that I would like to highlight here 'just using a lower 95% interval bound is very arbitrary'. I think you can do better.

For a simple case: assume that individuals performance is normal distributed and that the individual mean level is itself also normal distributed. Then we can consider a ranking based on estimates of the individual means. This can be done with some Bayesian technique or with an random effects model. In both cases you will get that the estimates for players with little data will be closer to the general mean.

Instead of computing a single random effects model or Bayesian model, a simpler practical way to continuously update a ranking would be to use some elo-rating system (whose mechanism will somewhat boil down to a sort of Bayesian model and probably you might even show that it is equivalent)

Elo-rating type of systems considers a prior distribution for the level of players and based on the outcome, win or lose, these distributions are updated.

In bridge it will be a bit more complex than a win or loose in a game of two players, but the mechanism of elo rating is very simple and in this way you might also be able to incorporate the complex parts of Bridge tournaments.

• Instead of win/lose you have a score. But, basically you just need some function that predicts the outcome based on a difference in elo rating level. In chess this is a logistic function, for bridge you could convert the prior distribution of performance level difference into points scored according to some function that can start of a bit arbitrary, and eventually be updated based on observed distribution of points for given difference in Elo rating.

• The links from Kjetil's answer propose to give individuals their own ratings and average them in some way to get an Elo rating for the couples.

• Another thing that you can add to this (which, I believe, is not mentioned in those links) is that in duplicate bridge multiple pairs of pairs play the same hands and you can compare the relative performance for the group as a whole. E.g. if a couple A with Elo 1800 plays against couple B with Elo 1950 and couple C with Elo 1900 plays against couple D with Elo 2000, then you expect that couple B is gonna score better than couple D. Based on the observed difference in score.

  Instead of making computations for each deal, it might be easier to look at the end results of an entire tournent and use the total scores of the couples. Then based on the observed scores and the expected scores (e.g. some Gaussian distribution based on the ELO and the number of deals) you compute a new ELO rating.