# Calculating variance of a random variable with different probability of occurring

I want to calculate the variance of sample mean of an iid random sample $$X_{1}, X_{2}, \cdots X_{n}$$ of the random variable $$X$$ which is distributed as $$N(\mu, \sigma^{2})$$ with probability $$(1-\delta)$$ and $$f(x)$$ with probability $$\delta$$.

The pdf $$f(x)$$ is any density with mean $$\theta$$ and variance $$\tau^{2}$$.

Statistical inference (2nd edition) by Casella and Berger says

$$Var(\bar{X}) = (1-\delta) \frac{\sigma^{2}}{n} + \delta \frac{\tau^{2}}{n} + \frac{\delta(1-\delta)(\theta - \mu)^{2}}{n}$$

but I cannot understand how I can arrive here. All I can guess is that since $$\bar{X}$$ is a sample mean, $$Var(\bar{X}) = \frac{Var(X)}{n}$$, but I cannot seem to calculate the variance of $$X$$ in the first place. Originally, I thought I can write $$X$$ as a linear combination of two random variables, $$X = (1-\delta)Y_{1} + \delta Y_{2}$$, with $$Y_{1}$$ distributed as $$N(\mu, \sigma^{2})$$ and $$Y_{2}$$ distributed as $$f(x)$$, but this does not seem to be right. Can you explain how to calculate the correct variance as given above, and how it is different from a linear combination of two random variables?

Any help will be greatly appreciated. Thank you.

Let $$\mathcal T$$ be the type of $$X$$ (i.e. normal or not, $$\mathcal N,\mathcal N'$$). Using the law of total variance, we can write: $$\operatorname{var}(X)=\mathbb E[\operatorname{var}(X|\mathcal T)] + \operatorname{var}(\mathbb E[X|\mathcal T])$$

For the first term, $$\mathbb E[\operatorname{var}(X|\mathcal T)]=P(\mathcal N)\operatorname{var}(X|\mathcal N)+P(\mathcal N')\operatorname{var}(X|\mathcal N')=(1-\delta)\sigma^2+\delta\tau^2$$

For the second term, $$\mathbb E[X|\mathcal T]$$ is a function of $$\mathcal T$$, and it has two options (i.e. either $$\mu$$ with probability $$1-\delta$$ or $$\theta$$ with probability $$\delta$$). You can directly calculate the variance using this, but this is a scaled and shifted version of a Bernoulli RV, say $$Z$$. So,

$$f(\mathcal T)=(\theta-\mu)Z+\mu$$

If $$Y$$ is $$1$$, $$f(\mathcal T)$$ is $$\theta$$, else $$\mu$$. And, let $$Z$$ be $$1$$ with probability $$\delta$$. $$\operatorname{var}((\theta-\mu)Z+\mu)=(\theta-\mu)^2\operatorname{var}(Z)=(\theta-\mu)^2\delta(1-\delta)$$

If you substitute all of these, you get the reported expression.

Regarding the linear combination, you can do that for the PDFs: $$f_X(x)=(1-\delta) f_{Y_1}(x)+\delta f_{Y_2}(x)$$ And, you could calculate the variance using this expression as well. But, it'd be more cumbersome. The correct representation of the RV $$X$$ in terms of $$Y_i$$ is the following:

$$X=\begin{cases}Y_1&\text{with prob. } &1-\delta\\Y_2 & \text{with prob. } &\delta\end{cases}$$

• Thank you for your help! Found it very helpful. – Siho Park Dec 2 '20 at 13:16