Fisher information matrix for comparing two treatments

Question

This is an exercise from Larry Wasserman's book "All of Statistics". Unfortunately, there is no solution online.

The exercise is the following (quoting from Wasserman's book):

$n_1$ people are given treatment $1$ and $n_2$ people are given treatment $2$. Let $X_1$ be the number of people on treatment $1$ who respond favourably to the treatment and $X_2$ be the number of people on treatment $2$ who respond favourably. Assume $X_1 \sim Binomial(n_1,p_1)$ and $X_2 \sim Binomial(n_2, p_2)$. Let $\psi := p_1-p_2$.

First task was to find the MLE of $\psi$ which is just the $\hat{p}_1-\hat{p}_2$ where $\hat{p}_i$ is the MLE of $p_i$ by functional invariance of the MLE. The second task is to find the Fisher information matrix $I(p_1, p_2)$, where the generally $(i,j)$ entry $H_{i,j}$ is defined as the expectation of

$$ H_{i,j}=\frac{\partial^2 l_n}{\partial \theta_i\partial \theta_j}$$

and $l_n := \sum_{i = 1}^n \log{f(X_i;\theta)}$. In our case $\theta_1 = p_1$ and $\theta_2 = p_2$, i.e. the Fisher information matrix is a $2\times 2$ matrix. I'm puzzled about the different $n$. Is in this case $n=2$ for $X_1$ and $X_2$ or is it $n=n_1+n_2$? I think it is $2$, is this correct?

So lets calculate the matrix entries. For this note

$$l_n = \sum_{i=1}^2\log{\binom{n_i}{x_i}p_i^{x_i}(1-p_i)^{n_i-x_i}}$$

since we will take partial derivative wrt to $p_i$ we can forget about the binomial coefficient, i.e.

$$l_n = \sum_{i=1}^2x_i\log{p_i}+(n_i-x_i)\log{(1-p_i)}$$

So we get

$$H_{ii}=-\frac{x_i}{p_i^2}+\frac{n_i-x_i}{(1-p_i)^2}$$ and $$H_{ij}=H_{ji}=\frac{x_i}{p_i}-\frac{n_i-x_i}{1-p_i}+\frac{x_j}{p_j}-\frac{n_j-x_j}{1-p_j}$$

So taking expectation we get

$$E_{p_ip_i}[H_{ii}]=-\frac{n_ip_i}{p_i^2}+\frac{n_i-n_ip_i}{(1-p_i)^2}=-\frac{n_i}{p_i}+\frac{n_i}{1-p_i}$$ and $$E_{p_ip_j}[H_{ij}]=E_{p_ip_j}[H_{ji}]=0$$

Is this correct?

Answer 1

I'm puzzled about the different n. Is in this case n=2 for X1 and X2 or is it n=n1+n2? I think it is 2, is this correct?

Your data is made of $(x_1,x_2)$ which is a single observation of a random vector $(X_1,X_2)$ where $$X_1\sim\mathcal{B}(n_1,p_1)\qquad X_2\sim\mathcal{B}(n_2,p_2)$$ with $X_1$ independent from $X_2$. Hence $n=1$.

$$H_{ii}=-\frac{x_i}{p_i^2}+\frac{n_i-x_i}{(1-p_i)^2}$$

This is not correct: Since $$l_n = x_1\log{p_1}+(n_1-x_1)\log{(1-p_1)}+x_2\log{p_2}+(n_2-x_2)\log{(1-p_2)}$$ the second derivative $$H_{11}=\dfrac{\partial^2 l_n}{\partial p_1^2}$$ is given by $$H_{11}=\dfrac{\partial^2 l_n}{\partial p_1^2}=\dfrac{\partial }{\partial p_1} \left\{\dfrac{x_1}{p_1} - \dfrac{n_1-x_1}{1-p_1}\right\}=-\dfrac{x_1}{p_1}^2-\dfrac{n_1-x_1}{(1-p_1)^2}$$ with expectation $$\mathbb{E}[H_{11}]= -\dfrac{\mathbb{E}[x_1]}{p_1^2}-\dfrac{\mathbb{E}[n_1-x_1]}{(1-p_1)^2}=-\dfrac{n_1}{p_1}-\dfrac{n_1}{1-p_1}$$

$$H_{ij}=H_{ji}=\frac{x_i}{p_i}-\frac{n_i-x_i}{1-p_i}+\frac{x_j}{p_j}-\frac{n_j-x_j}{1-p_j}$$

This is not correct: Since $$l_n = x_1\log{p_1}+(n_1-x_1)\log{(1-p_1)}+x_2\log{p_2}+(n_2-x_2)\log{(1-p_2)}$$ the second derivative $$H_{12}=\dfrac{\partial^2 l_n}{\partial p_1\partial p_2}$$ is given by $$H_{12}=\dfrac{\partial^2 l_n}{\partial p_1\partial p_2}=\dfrac{\partial }{\partial p_1} \left\{\dfrac{x_2}{p_2} - \dfrac{n_2-x_2}{1-p_2}\right\}=0$$ since the term between braces does not depend on $p_1$. That the expectation of $H_{12}=H_{21}$ is indeed zero is coherent with the fact that $X_1$ and $X_2$ are independent.