3
$\begingroup$

I want to calculate the variance of sample mean of an iid random sample $X_{1}, X_{2}, \cdots X_{n}$ of the random variable $X$ which is distributed as $N(\mu, \sigma^{2})$ with probability $(1-\delta)$ and $f(x)$ with probability $\delta$.

The pdf $f(x)$ is any density with mean $\theta$ and variance $\tau^{2}$.

Statistical inference (2nd edition) by Casella and Berger says

$$Var(\bar{X}) = (1-\delta) \frac{\sigma^{2}}{n} + \delta \frac{\tau^{2}}{n} + \frac{\delta(1-\delta)(\theta - \mu)^{2}}{n}$$

but I cannot understand how I can arrive here. All I can guess is that since $\bar{X}$ is a sample mean, $Var(\bar{X}) = \frac{Var(X)}{n}$, but I cannot seem to calculate the variance of $X$ in the first place. Originally, I thought I can write $X$ as a linear combination of two random variables, $X = (1-\delta)Y_{1} + \delta Y_{2}$, with $Y_{1}$ distributed as $N(\mu, \sigma^{2})$ and $Y_{2}$ distributed as $f(x)$, but this does not seem to be right. Can you explain how to calculate the correct variance as given above, and how it is different from a linear combination of two random variables?

Any help will be greatly appreciated. Thank you.

$\endgroup$
2
$\begingroup$

Let $\mathcal T$ be the type of $X$ (i.e. normal or not, $\mathcal N,\mathcal N'$). Using the law of total variance, we can write: $$\operatorname{var}(X)=\mathbb E[\operatorname{var}(X|\mathcal T)] + \operatorname{var}(\mathbb E[X|\mathcal T])$$

For the first term, $$\mathbb E[\operatorname{var}(X|\mathcal T)]=P(\mathcal N)\operatorname{var}(X|\mathcal N)+P(\mathcal N')\operatorname{var}(X|\mathcal N')=(1-\delta)\sigma^2+\delta\tau^2$$

For the second term, $\mathbb E[X|\mathcal T]$ is a function of $\mathcal T$, and it has two options (i.e. either $\mu$ with probability $1-\delta$ or $\theta$ with probability $\delta$). You can directly calculate the variance using this, but this is a scaled and shifted version of a Bernoulli RV, say $Z$. So,

$$f(\mathcal T)=(\theta-\mu)Z+\mu$$

If $Y$ is $1$, $f(\mathcal T)$ is $\theta$, else $\mu$. And, let $Z$ be $1$ with probability $\delta$. $$\operatorname{var}((\theta-\mu)Z+\mu)=(\theta-\mu)^2\operatorname{var}(Z)=(\theta-\mu)^2\delta(1-\delta)$$

If you substitute all of these, you get the reported expression.

Regarding the linear combination, you can do that for the PDFs: $$f_X(x)=(1-\delta) f_{Y_1}(x)+\delta f_{Y_2}(x)$$ And, you could calculate the variance using this expression as well. But, it'd be more cumbersome. The correct representation of the RV $X$ in terms of $Y_i$ is the following:

$$X=\begin{cases}Y_1&\text{with prob. } &1-\delta\\Y_2 & \text{with prob. } &\delta\end{cases}$$

$\endgroup$
1
  • $\begingroup$ Thank you for your help! Found it very helpful. $\endgroup$
    – Siho Park
    Dec 2 '20 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.