4
$\begingroup$

This is an exercise from Larry Wasserman's book "All of Statistics". Unfortunately, there is no solution online.

The exercise is the following (quoting from Wasserman's book):

$n_1$ people are given treatment $1$ and $n_2$ people are given treatment $2$. Let $X_1$ be the number of people on treatment $1$ who respond favourably to the treatment and $X_2$ be the number of people on treatment $2$ who respond favourably. Assume $X_1 \sim Binomial(n_1,p_1)$ and $X_2 \sim Binomial(n_2, p_2)$. Let $\psi := p_1-p_2$.

First task was to find the MLE of $\psi$ which is just the $\hat{p}_1-\hat{p}_2$ where $\hat{p}_i$ is the MLE of $p_i$ by functional invariance of the MLE. The second task is to find the Fisher information matrix $I(p_1, p_2)$, where the generally $(i,j)$ entry $H_{i,j}$ is defined as the expectation of

$$ H_{i,j}=\frac{\partial^2 l_n}{\partial \theta_i\partial \theta_j}$$

and $l_n := \sum_{i = 1}^n \log{f(X_i;\theta)}$. In our case $\theta_1 = p_1$ and $\theta_2 = p_2$, i.e. the Fisher information matrix is a $2\times 2$ matrix. I'm puzzled about the different $n$. Is in this case $n=2$ for $X_1$ and $X_2$ or is it $n=n_1+n_2$? I think it is $2$, is this correct?

So lets calculate the matrix entries. For this note

$$l_n = \sum_{i=1}^2\log{\binom{n_i}{x_i}p_i^{x_i}(1-p_i)^{n_i-x_i}}$$

since we will take partial derivative wrt to $p_i$ we can forget about the binomial coefficient, i.e.

$$l_n = \sum_{i=1}^2x_i\log{p_i}+(n_i-x_i)\log{(1-p_i)}$$

So we get

$$H_{ii}=-\frac{x_i}{p_i^2}+\frac{n_i-x_i}{(1-p_i)^2}$$ and $$H_{ij}=H_{ji}=\frac{x_i}{p_i}-\frac{n_i-x_i}{1-p_i}+\frac{x_j}{p_j}-\frac{n_j-x_j}{1-p_j}$$

So taking expectation we get

$$E_{p_ip_i}[H_{ii}]=-\frac{n_ip_i}{p_i^2}+\frac{n_i-n_ip_i}{(1-p_i)^2}=-\frac{n_i}{p_i}+\frac{n_i}{1-p_i}$$ and $$E_{p_ip_j}[H_{ij}]=E_{p_ip_j}[H_{ji}]=0$$

Is this correct?

$\endgroup$
  • $\begingroup$ Are $N_1$ and $N_2$ (abuse of notation; using the sample size to denote the sample) independently sampled, and is any part of $N_1$ ($N_2$) used to compute $\hat{p}_2$ ($\hat{p}_1$)? $\endgroup$ – tchakravarty Oct 11 '15 at 16:30
4
$\begingroup$

I'm puzzled about the different n. Is in this case n=2 for X1 and X2 or is it n=n1+n2? I think it is 2, is this correct?

Your data is made of $(x_1,x_2)$ which is a single observation of a random vector $(X_1,X_2)$ where $$X_1\sim\mathcal{B}(n_1,p_1)\qquad X_2\sim\mathcal{B}(n_2,p_2)$$ with $X_1$ independent from $X_2$. Hence $n=1$.

$$H_{ii}=-\frac{x_i}{p_i^2}+\frac{n_i-x_i}{(1-p_i)^2}$$

This is not correct: Since $$l_n = x_1\log{p_1}+(n_1-x_1)\log{(1-p_1)}+x_2\log{p_2}+(n_2-x_2)\log{(1-p_2)}$$ the second derivative $$H_{11}=\dfrac{\partial^2 l_n}{\partial p_1^2}$$ is given by $$H_{11}=\dfrac{\partial^2 l_n}{\partial p_1^2}=\dfrac{\partial }{\partial p_1} \left\{\dfrac{x_1}{p_1} - \dfrac{n_1-x_1}{1-p_1}\right\}=-\dfrac{x_1}{p_1}^2-\dfrac{n_1-x_1}{(1-p_1)^2}$$ with expectation $$\mathbb{E}[H_{11}]= -\dfrac{\mathbb{E}[x_1]}{p_1^2}-\dfrac{\mathbb{E}[n_1-x_1]}{(1-p_1)^2}=-\dfrac{n_1}{p_1}-\dfrac{n_1}{1-p_1}$$

$$H_{ij}=H_{ji}=\frac{x_i}{p_i}-\frac{n_i-x_i}{1-p_i}+\frac{x_j}{p_j}-\frac{n_j-x_j}{1-p_j}$$

This is not correct: Since $$l_n = x_1\log{p_1}+(n_1-x_1)\log{(1-p_1)}+x_2\log{p_2}+(n_2-x_2)\log{(1-p_2)}$$ the second derivative $$H_{12}=\dfrac{\partial^2 l_n}{\partial p_1\partial p_2}$$ is given by $$H_{12}=\dfrac{\partial^2 l_n}{\partial p_1\partial p_2}=\dfrac{\partial }{\partial p_1} \left\{\dfrac{x_2}{p_2} - \dfrac{n_2-x_2}{1-p_2}\right\}=0$$ since the term between braces does not depend on $p_1$. That the expectation of $H_{12}=H_{21}$ is indeed zero is coherent with the fact that $X_1$ and $X_2$ are independent.

$\endgroup$
  • 1
    $\begingroup$ Thanks, yes there was the sign error. Many thanks for your answer. $\endgroup$ – math Oct 11 '15 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.