How to diagonalize a covariance matrix into Identity matrix Given a symmetric, positive definite, covariance matrix $S$,
how can I calculate a matrix $C$ such that 
$CSC^T$  is the Identity matrix $I$? 
 A: Since I haven't found a duplicate and nobody else has pointed one out, I have expanded my comment into an answer and added an example to show you how it works.
Any invertible matrix $A$ for which $AA^T=S$ will work by taking $C=A^{−1}$ .
So for example, Cholesky decomposition is one common way to do it - it's a simple algorithm for finding a lower triangular $A$ (or equivalently, in many implementations, an upper triangular $A^T$). That is, it finds $S=LL^T$, whence $L^{−1}S(L^{−1})^T=I$.
Example:
Consider the matrix
$$S=\left(\begin{matrix} 4 & 1 \\ 1 & 2.5 \end{matrix}\right) $$
We write $S = LL^T$, which implies that
$$s_{11} = l_{11}^2$$
That is, $l_{11}=2$ (we'll take positive square roots each time to make our diagonals on $L$ positive).
It also implies that $s_{21} = l_{21}l_{11}$, so $l_{21} = s_{21}/l_{11}$, i.e. $l_{21} = 1/2$. Then we see that $s_{22} = l_{21}^2+l_{22}^2$, so $l_{22}^2 = s_{22} - l_{21}^2 = 2.5 - 0.5^2 = 2.25$, in which case $l_{22} = 1.5$, so
$$L=\left(\begin{matrix} 2 & 0 \\ 0.5 & 1.5 \end{matrix}\right) $$
You can confirm that  
$$C = L^{-1}=\left(\begin{matrix} \frac{1}{2} & 0 \\ -\frac{1}{6} & \frac{2}{3} \end{matrix}\right) $$
and that $CSC^T = I$. 
In practice we don't invert $L$, however, but solve a linear system, it's more stable.
Another possibility is to use Singular Value Decomposition to find a matrix to do the diagonalizing, and then scale by the singular values.
But a number of other choices are available. (As whuber points out, a very large number.)
In practice, frequently we know $S$ is of the form $X^TX$ for some $X$. In that case, it's usually better to work on $X$ than $S$, via the QR decomposition.
