# On approximating the MSE of an estimator

I'm trying to approximate the MSE of an estimator through simulation, in particular estimators of the form $$\hat{\theta} = \sum_{i=1}^N w_i X_i$$

Where $X = \{X_1,...,X_N\}$ are i.i.d. samples from some distribution and $\sum_{i} w_i = 1$ but the $w_i$ are not independent of $X$, and N is large.

I also happen to know the true $\theta = \mathbb{E}[X_i]$

Currently, I am running a large number of simulations $M$ and computing MSE as $$\frac{1}{M} \sum_{j=1}^M (\hat{\theta}_j - \theta)^2$$

But this is rather expensive, I'm wondering if there is a better way.

edit: In case some additional context may be useful.

Suppose that each $X_j$ is actually distributed from one of $K << N$ distributions and it is known which one, and further, that the true variance of the distribution, $\sigma_j^2$, is known (for evaluation purposes only).

Let $s_j^2$ denote the sample variance for the distribution of $X_j$, computed using all samples sharing that distribution. Then, $w_i = (s_i^2 + c_i) / \sum_j (s_j^2+c_j)$. In other words, the dependence between $X_j$ and $w_j$ is certainly not negligible.

• Is there a set way the weights depend on the samples, or is it arbitrary? – Accidental Statistician Nov 7 '14 at 16:16
• @AccidentalStatistician I edited the question to answer this – fairidox Nov 7 '14 at 19:22

Since the $X_i$ are identically distributed you can denote $\sigma^2 = \mathrm{Var}(X_i)$. Now, \begin{align*} \mathrm{Var} \, \hat\theta &= \mathrm{Var} \, \sum_{i=1}^N w_i X_i \\ &= \sum_{i=1}^N w_i^2 \mathrm{Var} \,X_i \qquad \text{by independence,}\\ &= \sigma^2\sum_{i=1}^N w_i^2 \end{align*} Meanwhile, \begin{align*} \frac1N \sum_{j=1}^M (\hat\theta_j - \theta)^2 \underset{LLN}{\longrightarrow} \mathbb E [ (\hat\theta - \theta)^2] = \mathrm{Var}\,\hat\theta \end{align*} Since $\mathbb E \, \hat\theta = \theta$. Hope this helps.
• The variance calculation errs by assuming the $w_i$ are constants, whereas the question asserts they are random variables "not independent of" the $X_i$. What you need to apply instead is a formula for the variance of an inner product of random vectors $(w_i)$ and $(X_i)$. – whuber Nov 7 '14 at 19:25