2
$\begingroup$

Suppose we know $\mu_1,\mu_2, \dots,\mu_n $ which are true values. Suppose we simulate data $x_{i1},\dots, x_{in}$ from some model or distribution involving, $\mu_i$. Using this we want to calculate the Mean Squared Error(MSE), thinking that the mean is a good estimate for $\mu_i$.

$$\hat{MSE}=\frac{1}{n}\sum^n_{i=1} (\mu_i-\bar{x_i})^2$$

My question is What is the variance of the estimate of this MSE and how we calculate it, analytically and computationally? I wonder whether the central limit theorem can be applied here.

Computationally, is it wise to just simulate the entire experiment $M$ times and compute it's variance. i.e. For simulation $M=1$, generate $x_{i1},\dots, x_{in}$ for each $i$,compute $\bar{x}_i$ for each $i$, compute $MSE$, repeat from start. We obtain $M$ different MSE estimates and so can compute the sample variance of that. This would be known as the variance or monte-carlo variance.

Improve this question
$\endgroup$
2
  • $\begingroup$ I think that the process you describe on the computational side is fine. For the analytical side, I think you can write the variance of the function you have already written as a function of the data (so basically a formula that is a variance of MSEs). I am not sure there needs to be much magic here. $\endgroup$
    – Deathkill14
    Mar 23, 2017 at 5:28
  • $\begingroup$ @Deathkill14 Thanks. Sorry, I think I may not have been clear. What I mean in "analytical side" is that I want to use some sort of central limit theorem, such that I can obtain $Var(\hat{MSE})$ only for 1 simulation. In the SEPARATE context of generating $\bar{x}$ from i.i.d $x's$ instead of the case of MSE which is not necessarily i.i.d, I can just take $Var(\bar{x})=\frac{\sigma^2}{n}$ where\ $ \sigma^2$ is the true variance(common) for each disrtibution generating $x's$ or can be estimated by taking the variance of all these samples. $\endgroup$
    – tintinthong
    Mar 23, 2017 at 9:46

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.