Use Bayesian modeling & PyMC to find players with similar skill levels to match for game I'm trying to learn Bayesian modeling and PyMC and I'm working on building a match-making service for an online game that avoids pitting two extremely unequal opponents against one another. What I'd like is to estimate the probability of player X winning vs. player Y given their match history in the game and then use this to prohibit very skilled players from mopping the floor with newer or weaker players.
It seems that the actual probability of player X defeating player Y can be measured with a Bernoulli variable but I'm struggling with how to incorporate the players' previous matches. Furthermore, some players will have played more matches than others and the distributions of their performance will be flatter and more uncertain. Then, I may also want to incorporate other performance variables that could influence performance. However, I'm still struggling with how to even set up a basic example where the two performance variables of the players influence the single probability that player X wins. Any suggestions would be much appreciated.
 A: It sounds like you'd be interested in the Bradley–Terry model, which models the probability of a player winning a match against another player. It is generally estimated in a fashion that takes into account each player's win–loss records against all other players. A detailed example of its use to model performance in a chess tournament, in a Bayesian context, can be found in Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2004), Bayesian data analysis (2nd ed.), Boca Raton, FL: Chapman & Hall/CRC, ISBN 1-58488-388-X, pages 431–433. Gelman et al. cite Glickman, M. E. (1993), Paired comparison models with time-varying parameters (Ph.D. thesis), Department of Statistics, Harvard University, http://www.glicko.net/research/thesis.pdf .
This said, beware that MCMC is slow and computationally intensive, almost certainly prohibitively so for real-time use on a web server. Even MLE, which can be thought of as an approximation to a full Bayesian analysis, may be too slow. Perhaps you can find an approximation that is computationally much simpler but gets most of what you want. In any case, you may be able to find a way to reuse partial results so that only a few computations need to be done per candidate matchup.
A: This in not a PyMC answer, but here is a piece of code inspired from https://probmods.org/chapters/03-conditioning.html#example-reasoning-about-tug-of-war 
It is in WebPPL, implementing a MCMC way to find player's strenght given a tournament results.
var model = function() {
  var strength = mem(function (person) {return uniform(0, 1)})
  var winner = function (player1, player2) {
    strength(player1) > strength(player2) ? player1 : player2 }
  var beat = function(player1,player2){winner(player1,player2) == player1}

  condition(beat('bob', 'mary'))
  return(strength('bob'))
}

var dist = Infer({method: 'MCMC', kernel: 'MH', samples: 25000},
            model)
console.log('Expected : ' + expectation(dist))
viz(dist)

The online example also introduces teams made by several players and a parameter of a player strength : he is perhaps lazy.
BTW, the probmods.org online book is a fascinating piece.
About PyMC, I'm as you, I don't have any clue to how implement this, especially how to formalize observations.
