I am trying to prove a lemma. Consider $X_1,.,X_n$ to be i.i.d random variables. Show that, in particular: $$E\|\hat{\mu}-\mu \|^2=\frac{1}{n}\text{trace}(\Sigma)$$

I have the following partial solution:

$$E\|\hat{\mu}-\mu\|^2=E(\hat{\mu}-\mu)^T(\hat{\mu}-\mu)$$, since $\|x\|=x^Tx$, now using the definition of $\hat\mu=\frac{1}{n}\displaystyle\sum_{i=1}^nX_i$ I can write the last expression as: $$E(\frac{1}{n}\sum_{i=1}^n(X_i-\mu)^T(\frac{1}{n}\sum_{j=1}^nX_j-\mu))$$ Now, I put the factors $\frac{1}{n}$ outside the expectation $$\frac{1}{n^2}E(\sum_{i=1}^n(X_i-\mu)^T(\sum_{j=1}^n(X_j-\mu))$$

Since the random variables $X_i$ are i.i.d it means that in the cross products gotten solving the previous expression $i\neq j$, I will get something like $$\frac{1}{n^2}E(\sum_{i,j=1}^n(X_i-\mu)^T(X_j-\mu))$$

that is equal to: $$\frac{1}{n^2}(\sum_{i}^nE(X_i-\mu)^T(X_i-\mu))$$

now, according to the definition of matrix $\Sigma_{XY}=E[XY^T-\mu_X\mu_Y]=Cov(X,Y)$ in this case the last expression is equal to $Cov(X_i,X_i)=E[X_i^TX_i-\mu^t\mu]=Var(X_i)$ then the previous expression is equal to: $$=\frac{1}{n^2}\sum_{i=1}^nVar(X_i)=\frac{1}{n^2}trace(\Sigma)$$ That is not what I need to show, could you explain me what is wrong in my solution?

• You are likely to find a much simpler solution by writing out the case $n=2$ in terms of the components of the vectors and matrices. – whuber Feb 1 '18 at 15:19
• I cant conclude, could you help me: I consider $n=2$, then I got: $\frac{1}{4}(EX_1^T-\mu^T\mu+EX_2^TX_2-\mu^T\mu)$ that is equal to $\frac{1}{4}(Var(X_1)+Var(X_2))=\frac{1}{4}(\Sigma_{11}+\Sigma_{22})=\frac{1}{4}(trace(\Sigma))$ – Boris Feb 1 '18 at 15:26
• I think a lot of people are giving great solutions. But the problem is you are confusion the sample size and the dimension size. $E(X_i-\mu)^\top (X_i-\mu)= trace(\Sigma)$ So you add up n trace of sigmas. – Josh Feb 1 '18 at 19:12

The third line uses $\textrm{trace}(AB) = \textrm{trace}(BA)$ (when both products can be performed) and the fourth line the fact that the order of the expectation and trace operators can be interchanged.
• But why,$$(\hat\mu-\mu)^T(\hat\mu-\mu)=trace(\hat\mu-\mu)^T(\hat\mu-\mu)$$? – Boris Feb 1 '18 at 17:03
• and why $E(\hat\mu-\mu)(\hat\mu-\mu)^T=\frac{1}{n}\Sigma$? That parts I dont have clear. – Boris Feb 1 '18 at 17:06
• Boris, for your first question, it helps to think about the dimensions of the matrix, or to work out a simple example. $(\hat{\mu}-\mu)$ is a $n \times 1$ vector. Obviously, the transpose is $1 \times n$. And when you multiply the $1 \times n$ vector with a $n \times 1$ vector, you get $\sum_{i=1}^n$ terms, which is the same as the sum of the terms along the diagonal of the $n \times n$ matrix $(\hat{\mu} - \mu)^T(\hat{\mu} - \mu)$. And the sum of the diagonal is... the trace! Try it out with an example and you'll see. – A. G. Feb 1 '18 at 17:16
• Doesn't the third line follow directly from the first more obviously than the second line? I wonder why you need the second line at all. You're using $x^T x = \text{trace}(x x^T),$ right? – Bridgeburners Feb 1 '18 at 19:09