# What is the variance of an MLE for a trinomial distribution?

I am playing with the following trinomial (multinomial) distribution which can get values (a,b,c) with the probabilities: $(\theta^2, 2\theta(1-\theta), (1-\theta)^2)$.

Say I have n observations from this distribution, then it is easy to show that the MLE for estimating $\theta$ is: $\hat \theta_{MLE} = {2\#a+ \#b \over 2n}$. However, I am trying to find the variance of this MLE, and keep getting a negative variance (which means I have a mistake somewhere, but I can't find where).

(EDIT: the correct answer is marked below :) )

• Those are Hardy-Weinberg proportions for two allele, single locus genotype frequencies. Dec 6, 2014 at 1:52
• Defining your slot(?) notation -- 2#a + #b -- would be helpful. Dec 6, 2014 at 4:08

Edit: I believe this is now correct

Let $X_a$ and $X_b$ be the counts in categories $a$ and $b$. Consider first the case for a single observation ($n=1$):

• $\text{Var}(X_a) = \theta^2(1-\theta^2)$

• $\text{Var}(X_b) = 2\theta(1-\theta)(1-2\theta(1-\theta))$

• Since the categories are mutually exclusive, $E(X_aX_b)=0$, so $\text{Cov}(X_a,X_b)=-E(X_a)E(X_b)=-2\theta^3(1-\theta)$.

Hence $Var(2X_a+X_b)=4Var(X_a)+Var(X_b)+4Cov(X_a,X_b)\\ \hspace{2.96cm}=4\theta^2(1-\theta^2)+2\theta(1-\theta)(1-2\theta(1-\theta))-8\theta^3(1-\theta)\\ \hspace{2.96cm}=2\theta(1-\theta)\,[2\theta(1+\theta)+(1-2\theta(1-\theta))-4\theta^2]\\ \hspace{2.96cm}=2\theta(1-\theta)$

Hence, (since the variance for $n$ independent observations is $n$ times the variance of one), $Var(2X_a+X_b)=2n\theta(1-\theta)$.

Hence $\text{Var}(\frac{1}{2n}(2X_a+X_b)) = \frac{1}{4n^2}2n\theta(1-\theta)=\frac{1}{2n}\theta(1-\theta)$.

This is non-negative for $0\leq\theta\leq 1$, and simulations agree with this answer.

• Thanks Glen, I see my problem was with E(X_a X_b). Could you please help me understand why it is 0? (and/or what is the distribution of X_a | X_b ? ) - thinking of this more: I think it shouldn't be 0 (I mean, both counts have a non-zero probability of both getting a positive number) Dec 6, 2014 at 2:20
• Tal - this the advantage of looking at a single observation first. One observation can only be in one category at a time, so at least one of the counts is necessarily 0, so the product in that case is 0. When you have multiple observations, the covariance must be the sum of covariances. The expectation of a product of counts won't be zero in that case, but that doesn't make the argument invalid - the covariance of the sum must still be the sum of the covariances of the one-observation cases. Dec 6, 2014 at 2:30
• Well, let's say we have n=2. Then There is some chance of there being 1 a and 1 b. Hence, I suspect, E above should not be 0. Dec 6, 2014 at 2:32
• I (think I) found the error. Typing it now and soon will put it online. Dec 6, 2014 at 2:38
• Answer posted - if you have a handy simulation - could you please have a look to see if I got it right? Dec 6, 2014 at 2:57

Thanks to the discussion with Glen, I realized that (assume ${X_a}$ is the number of a's and ${X_b}$ is the count of b's):

$${X_a}|{X_b} \sim B\left( {n - {X_b},{p_L} = \frac{{{\theta ^2}}}{{{\theta ^2} + {{\left( {1 - \theta } \right)}^2}}}} \right)$$

Hence:

$$E\left( {{X_a}|{X_b}} \right) = \left( {n - {X_b}} \right)\frac{{{\theta ^2}}}{{{\theta ^2} + {{\left( {1 - \theta } \right)}^2}}}{\rm{ }}$$

Making:

$${\mathop{\rm cov}} \left( {{X_a},{X_b}} \right) = - 2n\left( {1 - \theta } \right){\theta ^3}$$

$$V\left( {{{\hat \theta }_{MLE}}} \right) = \frac{{\theta \left( {1 - \theta } \right)}}{{2n}}$$

:)

• Defining $X_L$ and $X_M$ would be helpful. Dec 6, 2014 at 4:07
• If $X_L$ is the count in $a$, your answer agrees with my simulations. Dec 6, 2014 at 8:38
• wolfies - done. Glen_b - cool :) Dec 6, 2014 at 9:40
• I finally spotted my own remaining error as well Dec 6, 2014 at 11:43

For $n=1$ one can easily check that $2X_a+X_b$ has the binomial distribution on $\{0,1,2\}$ with success probability $\theta$. Then the expectation and the variance easily follow for any $n$. More precisely, one also deduces that $2X_a+X_b$ has the binomial distribution on $\{0,\ldots,2n\}$ with success probability $\theta$.)

In fact, you can see the model as the one for the following experiment. Let $Y_i \sim Bin(2, \theta)$ modeling one observation. Then you observe $f(Y_i)$ with $f(0)=c, f(1)=b, f(2)=a$. The sum $Y_1 + \cdots + Y_n \sim Bin(2n,\theta)$ is a sufficient statistic and the mle of $\theta$ is $\frac{Y_1 + \cdots + Y_n}{2n}$, and $Y_1 + \cdots + Y_n = 0 \times \#c + 1\times \#b + 2\times \#a$.

• @TalGalili Updated answer to show how to derive the mle. Dec 7, 2014 at 17:21