1
$\begingroup$

Suppose that I want to compute $E[X+Y]$ using Monte Carlo simulation and compute the standard error. (Note: $X,Y$ are not necessarily independent) The standard way to do this is to

  1. Consider the estimator $\frac{1}{n}\sum_{k=1}^n(X_k+Y_k)$ where $X_k$ and $Y_k$ are iid random variables with distributions the same as $X,Y$, respectively
  2. Compute the variance $Var(X_1+Y_1) = Var(X_1)+Var(Y_1) + 2Cov(X_1,Y_1)$
  3. Compute the standard error as the square root of $\frac{1}{n}Var(X_1 + Y_1)$

I am wondering why the following approach is incorrect. The reason why I think it's incorrect is because I don't get the same formula as above but I don't understand why:

  1. Since $E[X+Y] = E[X]+E[Y]$, approximate $E[X]$ by $\frac{1}{n}\sum_{k=1}^n X_k$ and $E[Y]$ by $\frac{1}{n}\sum_{k=1}^n Y_k$
  2. Compute the standard error of each which are $\sqrt{\frac{1}{n}Var(X_1)}$ and $\sqrt{\frac{1}{n}Var(Y_1)}$
  3. Add the standard errors for $E[X]$ and for $E[Y]$

Clearly, the two approaches give different results. Can anyone help me clarify the issue?

$\endgroup$

1 Answer 1

0
$\begingroup$

Let $\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i$ and $\bar{Y}_n = \frac{1}{n} \sum_{i=1}^n Y_i$ .
If $$ Z_n := \bar{X}_n + \bar{Y}_n $$

Then, since $X_i$, $Y_i$ are iid,

\begin{align*} \text{Var}(Z_n) &= \text{Var}(\bar{X}_n ) + \text{Var}(\bar{Y}_n) \\ &= \frac{1}{n}\text{Var}(X_1 ) + \frac{1}{n}\text{Var}(Y_1 ) \end{align*}

Thus the standard error of $Z_n$ is

$$ \sqrt{\text{Var}(Z_n)} =\sqrt{ \frac{1}{n}\text{Var}(X_1 ) + \frac{1}{n}\text{Var}(Y_1 )} $$

Since $\sqrt{x+y} \leq \sqrt{x} + \sqrt{y}$, $$ \sqrt{\text{Var}(Z_n)} \leq \sqrt{ \frac{1}{n}\text{Var}(X_1 ) } + \sqrt{\frac{1}{n}\text{Var}(Y_1 )}$$ So in the case of indenpendent variables, the variance of the sum is the sum of the variances, but the standard error of the sum is not the sum of the standard errors.

For example if we take $\text{Var}(X_1 ) = \text{Var}(Y_1 ) = \sigma^2$ we have $$ \sqrt{\text{Var}(Z_n)} = \sqrt{ \frac{2}{n}\sigma^2} = \sqrt{2} \sqrt{\frac{\sigma^2}{n}} $$ while $$ \sqrt{ \frac{1}{n}\text{Var}(X_1 ) } + \sqrt{\frac{1}{n}\text{Var}(Y_1 )} = 2 \sqrt{\frac{\sigma^2}{n}} $$

$\endgroup$
4
  • $\begingroup$ Hi, thanks for the response. I edited my question. $X,Y$ are not necessarily independent. I'm wondering why the 2nd approach is incorrect, i.e. what is conceptually wrong about it? $\endgroup$
    – secondrate
    Commented Sep 12, 2020 at 11:50
  • $\begingroup$ (a) If $X,Y$ are not independent, then the variance of the sum is not the sum of the variances. (b) The second approach is incorrect because the square root is not a linear operator: $(3+4)\times 2=3\times 2+4\times 2$, but $\sqrt{9}+\sqrt{16}=7\ne \sqrt{9+16}=5$. $\endgroup$
    – Sergio
    Commented Sep 12, 2020 at 12:41
  • $\begingroup$ @Sergio: Thanks for the reply. What's not clear to me is -- why can't you examine the standard error of $E[X]$ and $E[Y]$ and add them up? Mathematically, I see why they're different. Conceptually, I don't understand why it is wrong to add the standard error of each considering that $E[X+Y] = E[X] + E[Y]$ so I presume that we can just add the error for $E[X]$ with the error for $E[Y]$ to get the error for $E[X+Y]$ $\endgroup$
    – secondrate
    Commented Sep 12, 2020 at 12:45
  • 1
    $\begingroup$ Because the standard error is based on the standard deviation, which is the square root of the variance. And the square root is not a linear operator. Unlike the variance (of independent variables), the standard deviation of the sum is not the sum of the standard deviations. $\endgroup$
    – Sergio
    Commented Sep 12, 2020 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.