# Help with PCA Question

The conventional model for probabilistic principal component analysis has a standard normal latent $$\vec{y}$$ and a loading matrix $$\Lambda$$:

$$P(\vec{y}) \sim N(\vec{0}, I)$$, $$P(\vec{x}|\vec{y}) \sim N(\Lambda \vec{y}, \psi I)$$

An alternative would be to draw $$\vec{y}$$ from a normal with diagonal covariance (say $$\Sigma$$, and then restrict $$\Lambda$$ to have orthonormal columns:

$$P(\vec{y}) \sim N(\vec{0}, \Gamma), P(\vec{x}|\vec{y}) \sim N(\Lambda \vec{y}, \psi I)$$ with $$\Gamma_{ij}=0$$ for $$i \neq j$$ and $$\Lambda^T \Lambda = I$$

where $$\Gamma_{ij} =\Gamma_{ji} =0$$ and $$\Lambda^T \Lambda=0$$.

Question: Show that this alternative model is equivalent to the standard one in the sense that it can model exactly the same set of possible marginal distributions.

I really have no idea what I am being asked to do here. I know that there is some equivalence of PCA models under rotation and scaling - should I be working in that direction, or am I supposed to integrate out the latent variable and show I get the same distribution? Or something else entirely? Help please.

lead to exactly the same marginal distributions.

Here are some hints. I will rename $$\Lambda$$ in the first model to $$A$$ to avoid confusion with the one that is to have orthonormal columns.
As you seem to have deduced, the distributions we get for $$x$$ are $$N(0,\psi I + \Lambda \Gamma \Lambda^T)$$ in one model and $$N(0,\psi I + AA^T)$$ in the other model.
Therefore, it suffices to show that matrices of the form $$\Lambda \Gamma \Lambda^T$$ (with $$\Lambda$$ having orthonormal columns and $$\Gamma$$ diagonal) and $$AA^T$$ are equivalent (either can be written in the other's form).
To help show that $$\Lambda \Gamma \Lambda^T$$ can be written as $$AA^T$$, utilise the square root of $$\Gamma$$. To help show that $$AA^T$$ can be written as $$\Lambda \Gamma \Lambda^T$$, utilise the spectral theorem and drop eigenvectors corresponding to $$0$$ eigenvalues (and drop these eigenvalues from the diagonal matrix from the spectral theorem).