Standardisation in PCA: 'scaling' and 'sphering' and the covariance matrices of $\mathbf{X}_{\text{scale}}$ and $\mathbf{X}_{\Sigma}$ I'm currently studying standardisation in PCA. This primarily involves the generalised standardisation concepts of scaling and sphering:

The scaled random vector and random sample are
$$\mathbf{X}_{\text{scale}} = \Sigma_{\text{diag}}^{-1/2} (\mathbf{X} - \mathbf{\mu}) \ \ \ \text{and} \ \ \ \mathbb{X}_{\text{scale}} = S_{\text{diag}}^{-1/2} (\mathbb{X} - \overline{\mathbf{X}})$$
The *sphered random vector and random sample are
$$\mathbf{X}_\Sigma = \Sigma^{-1/2} (\mathbf{X} - \mathbf{\mu}) \ \ \ \text{and} \ \ \ \mathbb{X}_\Sigma = S^{-1/2} (\mathbb{X} - \overline{\mathbf{X}})$$

This is further illustrated in this image:



I am then told the following:

Let $\mathbf{X} \sim (\mathbf{\mu}, \Sigma)$, and assume that $\Sigma$ has rank $d$. Write $\mathbf{X}_{\text{scale}}$ and $\mathbf{X}_{\Sigma}$ for the scaled and sphered random vectors. These vectors have the following properties:

*

*The expected values of $\mathbf{X}_{\text{scale}}$ and $\mathbf{X}_\Sigma$ are $\mathbf{0}$;

*the covariance matrix of $\mathbf{X}_{\text{scale}}$ is the matrix of correlation coefficient $R$; and

*the covariance matrix of $\mathbf{X}_{\Sigma}$ is the identity matrix $\mathbf{I}$.


I am fine with claim 1., but I'm struggling to see how claims 2. and 3. are true. So how are claims 2. and 3. are true?
 A: Let $X \sim \mathcal{N}(\mu, \Sigma)$ and let $\Sigma$ be Cholesky decomposed into $$\Sigma = \Sigma^{1/2} (\Sigma^{1/2})^\top,$$
where $\Sigma^{1/2}$ is a lower triangular matrix. This is possible since $\Sigma$ is symmetric and positive definite. Then the random variable
$$
Y = \Sigma^{-1/2}  (X - \mu)
$$
is also a Normal random variable, since it is an affine transformation of the normal random variable $X$. Then $Y$ has the expected value
$$\mathbb{E}(Y) =  \Sigma^{-1/2}  (\mathbb{E}(X) - \mu) = \Sigma^{-1/2}  (\mu - \mu) = 0$$
and the covariance is
$$
\mathrm{Cov}(Y) = \Sigma^{-1/2} \mathrm{Cov}(X)(\Sigma^{-1/2})^{\top} = \Sigma^{-1/2} \Sigma^{1/2} (\Sigma^{1/2})^\top(\Sigma^{-1/2})^{\top} = I.
$$
Thus, $Y \sim \mathcal{N}(0,I)$.
If $Y = D (X - \mu)$ for a diagonal matrix $D$ with the diagonal filled with $\sigma_1,\sigma_2, \dots, $ the covariance of $Y$ is
$$
\mathrm{Cov}(Y) = D\Sigma D = R,
$$
where each element of $R$ is
$$
R_{ij} = \begin{cases} 1, & i=j  \\
 \frac{\mathrm{Cov}(x_i,x_j)}{\sigma_j \sigma_j}, & i \neq j
\end{cases}
$$
Thus $R$ is a "normalized" version of $\Sigma$, as pointed out by ArnoV
A: By normalising each variable $X_i$ using the marginal means and variances
$$\tilde{X}_i = \dfrac{X_i-\mu_i}{\sigma_i}$$
You make sure that $\mathbb{E}[\tilde{X}_i] = 0$ and $\mathrm{Var}[\tilde{X}_i] = 1$. You however do not fully impact the covariance, as you treat each variable separately. The new covariance changes as
$$\tilde{\Sigma}_{ij} =\mathbb{E}[\tilde{X}_i \tilde{X}_j] = \dfrac{1}{\sigma_i\sigma_j}\mathbb{E}[X_iX_j] = \dfrac{\mathrm{Cov}(X_i,X_j)}{\sigma_i\sigma_j}$$
In order to whiten the covariance, you need to combine different dimensions to remove the interaction between $X_i$ and $X_j$. This can be done through a linear transformation $\mathrm{A}x$. The "best" matrix $\mathrm{A}$ can be found by computing the transformed covariance
$$\mathbb{E}\left[(\mathrm{A}x)(\mathrm{A}x)^\mathrm{T}\right] = \mathrm{A}\,\mathbb{E}\left[xx^\mathrm{T}\right]\mathrm{A}^\mathrm{T} = \mathrm{A}\Sigma\mathrm{A}^\mathrm{T} = \mathrm{A}\Sigma^{1/2}\Sigma^{1/2}\mathrm{A}^\mathrm{T}$$
You can see there that if $\Sigma^{1/2}$ is symmetric and invertible, the matrix $\mathrm{A}=\Sigma^{-1/2}$ is the perfect transformation !
If you need to know why the matrix $\Sigma^{1/2}$ exists please open another question, preferably in the math section of the website.
