How to find the Fisher information matrix for a Bradley-Terry model I'm using a Bradley-Terry model to rank items based on pairwise comparisons. The model estimates a score for each item $p_i = e^{\beta_i}$. I want to get an idea of the uncertainty of each items score.
This paper suggests that I can compute the Fisher Information matrix, invert it to get an approximation to the covariance matrix from which I can take the square root of the diagonal to get the standard error ...which sounds great! Unfortunately I'm not sure how to construct the Fisher Information matrix in this case. Could someone give me a few pointers?
 A: I don't know the full details of your problem, so I'm going to make some assumptions, and spell them out so you can correct me if I'm wrong. Also my solution is only partially complete, but I hope it can serve as a starting point.
I am assuming you are working with a discrete set of $n$ items, so $i \in \{1, 2, \ldots, n\}$, meaning you have an $n$-parameter model, and thus will have an $n \times n$ Fisher matrix. Also, based on the two links you gave, the probability mass function for your model is
$$p_{ij} = \frac{p_i}{p_i + p_j},$$
which is a 2-dimensional distribution. If we define $\ell_{ij} = \log p_{ij}$, then the Fisher matrix will be given by
$$F_{ij} = \sum_{a=1}^n \sum_{b=1}^n \frac{\partial \ell_{ab}}{\partial i} \frac{\partial \ell_{ab}}{\partial j} p_{ab}$$
Now, I'm not really sure how to differentiate $\ell_{ab}$ wrt $i$ and $j$, so my answer falls flat of a complete solution here.
However, assuming you figure out this step, there's still the matter of the Cramér–Rao inequality, which states that the inverse Fisher matrix is only a lower bound on the covariance matrix, assuming you are using an unbiased estimator. You'll have to convince yourself of whether or not your estimator is unbiased, and if it meets the equality in the Cramér–Rao inequality.
