What is the variance of an MLE for a trinomial distribution?

Question

I am playing with the following trinomial (multinomial) distribution which can get values (a,b,c) with the probabilities: $(\theta^2, 2\theta(1-\theta), (1-\theta)^2)$.

Say I have n observations from this distribution, then it is easy to show that the MLE for estimating $\theta$ is: $\hat \theta_{MLE} = {2\#a+ \#b \over 2n}$. However, I am trying to find the variance of this MLE, and keep getting a negative variance (which means I have a mistake somewhere, but I can't find where).

(EDIT: the correct answer is marked below :) )

Answer 1

Edit: I believe this is now correct

Let $X_a$ and $X_b$ be the counts in categories $a$ and $b$. Consider first the case for a single observation ($n=1$):

• $\text{Var}(X_a) = \theta^2(1-\theta^2)$

• $\text{Var}(X_b) = 2\theta(1-\theta)(1-2\theta(1-\theta))$

• Since the categories are mutually exclusive, $E(X_aX_b)=0$, so $\text{Cov}(X_a,X_b)=-E(X_a)E(X_b)=-2\theta^3(1-\theta)$.

Hence $Var(2X_a+X_b)=4Var(X_a)+Var(X_b)+4Cov(X_a,X_b)\\ \hspace{2.96cm}=4\theta^2(1-\theta^2)+2\theta(1-\theta)(1-2\theta(1-\theta))-8\theta^3(1-\theta)\\ \hspace{2.96cm}=2\theta(1-\theta)\,[2\theta(1+\theta)+(1-2\theta(1-\theta))-4\theta^2]\\ \hspace{2.96cm}=2\theta(1-\theta)$

Hence, (since the variance for $n$ independent observations is $n$ times the variance of one), $Var(2X_a+X_b)=2n\theta(1-\theta)$.

Hence $\text{Var}(\frac{1}{2n}(2X_a+X_b)) = \frac{1}{4n^2}2n\theta(1-\theta)=\frac{1}{2n}\theta(1-\theta)$.

This is non-negative for $0\leq\theta\leq 1$, and simulations agree with this answer.

Answer 2

Thanks to the discussion with Glen, I realized that (assume ${X_a}$ is the number of a's and ${X_b}$ is the count of b's):

$${X_a}|{X_b} \sim B\left( {n - {X_b},{p_L} = \frac{{{\theta ^2}}}{{{\theta ^2} + {{\left( {1 - \theta } \right)}^2}}}} \right)$$

Hence:

$$E\left( {{X_a}|{X_b}} \right) = \left( {n - {X_b}} \right)\frac{{{\theta ^2}}}{{{\theta ^2} + {{\left( {1 - \theta } \right)}^2}}}{\rm{ }}$$

Making:

$${\mathop{\rm cov}} \left( {{X_a},{X_b}} \right) = - 2n\left( {1 - \theta } \right){\theta ^3}$$

Leading to:

$$V\left( {{{\hat \theta }_{MLE}}} \right) = \frac{{\theta \left( {1 - \theta } \right)}}{{2n}}$$

:)

Answer 3

For $n=1$ one can easily check that $2X_a+X_b$ has the binomial distribution on $\{0,1,2\}$ with success probability $\theta$. Then the expectation and the variance easily follow for any $n$. More precisely, one also deduces that $2X_a+X_b$ has the binomial distribution on $\{0,\ldots,2n\}$ with success probability $\theta$.)

In fact, you can see the model as the one for the following experiment. Let $Y_i \sim Bin(2, \theta)$ modeling one observation. Then you observe $f(Y_i)$ with $f(0)=c, f(1)=b, f(2)=a$. The sum $Y_1 + \cdots + Y_n \sim Bin(2n,\theta)$ is a sufficient statistic and the mle of $\theta$ is $\frac{Y_1 + \cdots + Y_n}{2n}$, and $Y_1 + \cdots + Y_n = 0 \times \#c + 1\times \#b + 2\times \#a$.