2
$\begingroup$

Suppose $X_1 \sim \text{binom}(p_1,n_1)$ and $X_2 \sim \text{binom}(p_2,n_2)$, where $X_1$ and $X_2$ are independent, and let $p = p_1 - p_2$. How can I prove that $\hat{p} = \hat{p}_1 - \hat{p}_2$? (Here, $\hat{\theta}$ denotes the MLE of $\theta$.) Initially, I tried to calculate the distribution of $X_1 + X_2$, but this turned out to be quite messy: Letting $Y = X_1 + X_2$, I found that for $y = 0,1,\ldots,n_1 + n_1$, \begin{align*} P(Y = y) &= P(X_1 + X_2 = y) \\[4pt] &= \sum_{x=0}^{n_1} P(X_1 + X_2 = y|X_1 = x)P(X_1 = x) \\[4pt] &= \sum_{x=0}^{n_1} P(x + X_2 = y) P(X_1 = x) \\[4pt] &= \sum_{x=0}^{n_1} P(X_2 = y - x) P(X_1 = x) \\[4pt] &= \sum_{x=0}^{n_1} {n_2 \choose y - x} p_2^{y-x} (1-p_2)^{n_2 - (y-x)} {n \choose n_1} p_1^{x} (1-p_1)^{n_1 - x} \end{align*} At this point I couldn't really recognize $Y$ as having a familiar distribution. Also, it's not clear to me if the likelihood function for $Y$ depends on $p_1$ and $p_2$ only through the difference $p = p_1 - p_2$, so I'm not clear on how to obtain the MLE for $p$.

I came across a post from 2015 on the same question, but it didn't have an answer. Also, one of the commenters on that post linked to this post on the invariance property of the MLE:

If $\hat{\theta}$ is the MLE of $\theta$, then for any function $\tau(\theta)$, the MLE of $\tau(\theta)$ is $\tau(\hat{\theta})$.

(I'm quoting Theorem 7.2.10 of Casella and Berger's Statistical Inference.) However, I do not see how this particular theorem applies to my question, since I'm interested in the MLE of a function of two parameters associated with two independent random variables. Now if there was an invariance theorem that said given independent random variables $X_1 \sim f(x_1|\theta_1)$ and $X_2 \sim g(x_2|\theta_2)$, we have $\widehat{\tau(\theta_1,\theta_2)} = \tau(\hat{\theta}_1,\hat{\theta}_2)$, then I would agree that $\hat{p} = \hat{p}_1 - \hat{p}_2$, but I don't believe that this is implied by the above theorem...Does such an invariance theorem exist? If not, how should I go about finding the MLE of $p$?

$\endgroup$
3
  • 4
    $\begingroup$ The parameter $\theta$ can be multivariate and the invariance property still holds! $\endgroup$
    – jbowman
    Apr 23 at 21:32
  • 3
    $\begingroup$ The invariance property ultimately is trivial: it reflects the fact that how you name the arguments of a function has no effect on what the values of that function are. Consequently, once you have found the extrema of a function (such as the likelihood), you can freely rename the arguments of (i.e., reparameterize) that function. $\endgroup$
    – whuber
    Apr 23 at 22:34
  • $\begingroup$ @jbowman: Thanks! If only I had read the very next page of Casella and Burger: "The invariance property of MLEs also holds in the multivariate case. There is nothing in the proof of Theorem 7.2.10 that precludes $\theta$ from being a vector. If the MLE of $(\theta_1,\ldots,\theta_k)$ is $(\hat{\theta}_1,\ldots,\hat{\theta}_k)$, then the MLE of $\tau(\theta_1,\ldots,\theta_k)$ is $\tau(\hat{\theta}_1,\ldots,\hat{\theta}_k)$." $\endgroup$
    – Leonidas
    May 2 at 19:14

1 Answer 1

1
$\begingroup$

I'll show you how the invariance property of the MLE applies to this case. Consider the joint distribution of the two random variables, which depends on the parameter vector $\mathbf{p} = (p_1,p_2)$. Suppose you find the MLE of this parameter vector via the joint distribution, and let's denote this MLE as:

$$\hat{\mathbf{p}}_\text{MLE} = (\hat{p}_1, \hat{p}_2).$$

Now, let's define the function $\tau$ so that $\tau(a,b) = a-b$. Applying this function to both the underlying parameter vector, and its MLE we get:

$$\begin{align} \tau(\mathbf{p}) = \tau(p_1, p_2) &= p_1 - p_2, \\[6pt] \tau(\hat{\mathbf{p}}_\text{MLE}) = \tau(\hat{p}_1, \hat{p}_2) &= \hat{p}_1 - \hat{p}_2. \\[6pt] \end{align}$$

Now, the invariance property of the MLE says that the MLE of $\tau(\mathbf{p})$ is $\tau(\hat{\mathbf{p}}_\text{MLE})$, which is equivalent to saying that the MLE of $p_1 - p_2$ is $\hat{p}_1 - \hat{p}_2$. (Note that your broader assertion that the joint distribution depends on $\mathbf{p}$ only through $p_1-p_2$ is false.)

$\endgroup$
1
  • 1
    $\begingroup$ Thank you! Somehow it hadn't immediately occurred to me that the invariance property still holds when $\theta$ is a vector. $\endgroup$
    – Leonidas
    May 2 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.