Suppose we know $\mu_1,\mu_2, \dots,\mu_n $ which are true values. Suppose we simulate data $x_{i1},\dots, x_{in}$ from some model or distribution involving, $\mu_i$. Using this we want to calculate the Mean Squared Error(MSE), thinking that the mean is a good estimate for $\mu_i$.

$$\hat{MSE}=\frac{1}{n}\sum^n_{i=1} (\mu_i-\bar{x_i})^2$$

My question is What is the variance of the estimate of this MSE and how we calculate it, analytically and computationally? I wonder whether the central limit theorem can be applied here.

Computationally, is it wise to just simulate the entire experiment $M$ times and compute it's variance. i.e. For simulation $M=1$, generate $x_{i1},\dots, x_{in}$ for each $i$,compute $\bar{x}_i$ for each $i$, compute $MSE$, repeat from start. We obtain $M$ different MSE estimates and so can compute the sample variance of that. This would be known as the variance or monte-carlo variance.

  • $\begingroup$ I think that the process you describe on the computational side is fine. For the analytical side, I think you can write the variance of the function you have already written as a function of the data (so basically a formula that is a variance of MSEs). I am not sure there needs to be much magic here. $\endgroup$ Mar 23, 2017 at 5:28
  • $\begingroup$ @Deathkill14 Thanks. Sorry, I think I may not have been clear. What I mean in "analytical side" is that I want to use some sort of central limit theorem, such that I can obtain $Var(\hat{MSE})$ only for 1 simulation. In the SEPARATE context of generating $\bar{x}$ from i.i.d $x's$ instead of the case of MSE which is not necessarily i.i.d, I can just take $Var(\bar{x})=\frac{\sigma^2}{n}$ where\ $ \sigma^2$ is the true variance(common) for each disrtibution generating $x's$ or can be estimated by taking the variance of all these samples. $\endgroup$ Mar 23, 2017 at 9:46


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.