Pairwise comparison with Bradley-Terry

Question

I am performing a pairwise comparison test for the perceived weight of objects. I want to estimate the difference between each pair, say, A - B. I suspect that the underlying distributions of A, and B, are Normal. However I am not sure, and I have no idea what their mean values or variances are. They could be described by any other symmetrical distribution (i.e. the weights of individual objects are perceived symmetrically around the mean perceived weight).

Using a Binomial model for the difference between each pair, it is easy to assume a Normal distribution for each team A, B, etc, and achieve satisfactory results.

If I instead want to use the Bradley-Terry model, this complicates things. The B-T model basically performs logistic regression, implying that the distribution for the difference between each pair A,B is a logistic distribution. If A-B follows a logistic distribution, A and B both follow Gumbel distributions (proved through convolution of Gumbel distributions). This is a problem, because I know that A and B follow symmetric distributions, whereas Gumbel is a skewed distribution. Would it be okay for me propose a normal approximation to the logistic distribution after use of the B-T model, in which case the distributions of A, B, etc. would all be normal? Or would this be counter-intuitive?

Answer 1

As you correctly point out, the Bradley-Terry model and the Thurstonian model can both be formulated as binary/binomial generalized linear models (GLMs) with either a logit or a probit link. As in other GLMs logit and probit typically yield almost identical results (see e.g., Difference between logit and probit models). So while you might prefer one or the other model from theoretical considerations, this often does not matter much for the interpretation ... at least as long as very extreme probabilities (close to zero or one) are avoided.

But you can also simply try this out. For example, the BradleyTerry2 package (Turner and Firth 2012, doi:10.18637/jss.v048.i09) supports setting both family = binomial(link = "logit") (default) or family = binomial(link = "probit").