2
$\begingroup$

I'm going through "Probabilistic Machine Learning: An Introduction" by Murphy (2021), and currently trying to do Ex. 3.1 (p. 80), which states:


Let $X \sim N(0, 1)$ and $Y = WX$, where $p(W = 1) = p(W =-1) = .5$. It is clear that X and Y are not independent, since Y is a function of X.

a. Show $Y \sim N(0, 1)$

b. Show $Cov[X, Y] = 0$ Thus X and Y are uncorrelated but dependent, even though they are Gaussian. Hint: use the definition of covariance:

$Cov[X,Y] = \mathbb{E}[XY]−\mathbb{E}[X]\mathbb{E}[Y]$ and $\mathbb{E}[XY ] = \mathbb{E}[\mathbb{E}[XY |W]]$


a) Now, conceptually, I get that $\mathbb{E}[W] = 0$ and that $Y = WX$ should be distributed $N(0, 1)$. But this is just because $W$ is such a simple distribution, and I would definitely be stuck if it were more complicated. So everything in between is done in my head because "it seems like it should be so", and not because I can derive it.

b) I also get that $\mathbb{E}[XY]$ and $\mathbb{E}[X]\mathbb{E}[Y]$ should be 0, therefore $Cov[EX]=0$, but also only on an abstract basis.

Could someone please guide me through this? This is not a homework question or any assignment, just trying to plough through the book by myself.

Thanks.

$\endgroup$
2
  • $\begingroup$ $Y$ is a mixture of two identically distributed symmetric variables and therefore has the same distribution. When those variables have a finite variance, the covariance must be zero by an analogous symmetry argument--no formulas are needed. See "Solution by circular symmetry" at stats.stackexchange.com/a/257919/919 for how one goes about making this kind of argument. $\endgroup$ – whuber Feb 23 at 13:27
  • 1
    $\begingroup$ @whuber Thanks for the explanation and the link. $\endgroup$ – Zlo Feb 23 at 13:44
2
$\begingroup$

Slight Generalization

The distribution of $Y$ can be factorized as $$p_Y(y) = \sum_{w} p_{Y,W}(y,w) = \sum_{w}p_W(w)p_{Y\mid W}(y\mid w) $$ If you denote $P(W=1)=q$ you obtain that $Y$ is distributed as $$ p_Y(y) = q\,p_X(x) + (1-q)\,p_X(-x)$$ However in your case you have a distribution which is symmetric about zero so that $p_X(x) = p_X(-x)$, therefore you obtain $p_Y = (q + 1 - q)p_X = p_X$.

Make it more general

You could make the problem more complex by having $W$ take $K$ non-zero values and use the same formulas. However you will need to derive the pdf of $Y\mid W=w$ which is equal to the pdf of $wX$: this is simply $\frac{1}{\lvert w \rvert}p_X(y/w)$.

Therefore you have that $Y$ is distributed as $$ p_Y(y) = \sum_{i=1}^K \dfrac{p(w_i)}{\lvert w_i \rvert}p_X(y/w_i)$$

$\endgroup$
1
  • $\begingroup$ Thanks, very helpful! $\endgroup$ – Zlo Feb 23 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.