0
$\begingroup$

I'm trying to approximate the MSE of an estimator through simulation, in particular estimators of the form $$ \hat{\theta} = \sum_{i=1}^N w_i X_i $$

Where $X = \{X_1,...,X_N\}$ are i.i.d. samples from some distribution and $\sum_{i} w_i = 1$ but the $w_i$ are not independent of $X$, and N is large.

I also happen to know the true $\theta = \mathbb{E}[X_i]$

Currently, I am running a large number of simulations $M$ and computing MSE as $$ \frac{1}{M} \sum_{j=1}^M (\hat{\theta}_j - \theta)^2 $$

But this is rather expensive, I'm wondering if there is a better way.

edit: In case some additional context may be useful.

Suppose that each $X_j$ is actually distributed from one of $K << N$ distributions and it is known which one, and further, that the true variance of the distribution, $\sigma_j^2$, is known (for evaluation purposes only).

Let $s_j^2$ denote the sample variance for the distribution of $X_j$, computed using all samples sharing that distribution. Then, $w_i = (s_i^2 + c_i) / \sum_j (s_j^2+c_j)$. In other words, the dependence between $X_j$ and $w_j$ is certainly not negligible.

$\endgroup$
  • 1
    $\begingroup$ Is there a set way the weights depend on the samples, or is it arbitrary? $\endgroup$ – Accidental Statistician Nov 7 '14 at 16:16
  • $\begingroup$ @AccidentalStatistician I edited the question to answer this $\endgroup$ – fairidox Nov 7 '14 at 19:22
0
$\begingroup$

Since the $X_i$ are identically distributed you can denote $\sigma^2 = \mathrm{Var}(X_i)$. Now, \begin{align*} \mathrm{Var} \, \hat\theta &= \mathrm{Var} \, \sum_{i=1}^N w_i X_i \\ &= \sum_{i=1}^N w_i^2 \mathrm{Var} \,X_i \qquad \text{by independence,}\\ &= \sigma^2\sum_{i=1}^N w_i^2 \end{align*} Meanwhile, \begin{align*} \frac1N \sum_{j=1}^M (\hat\theta_j - \theta)^2 \underset{LLN}{\longrightarrow} \mathbb E [ (\hat\theta - \theta)^2] = \mathrm{Var}\,\hat\theta \end{align*} Since $ \mathbb E \, \hat\theta = \theta $. Hope this helps.

$\endgroup$
  • 1
    $\begingroup$ "by independence" unfortunately I cannot assume this :) $\endgroup$ – fairidox Nov 7 '14 at 15:27
  • 2
    $\begingroup$ The variance calculation errs by assuming the $w_i$ are constants, whereas the question asserts they are random variables "not independent of" the $X_i$. What you need to apply instead is a formula for the variance of an inner product of random vectors $(w_i)$ and $(X_i)$. $\endgroup$ – whuber Nov 7 '14 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.