I have a $P \times K$ matrix $\mathbf{Z} = \begin{bmatrix}\mathbf{z_1} & \mathbf{z_1} & \cdots & \mathbf{z_K}\end{bmatrix}$. I want to whiten the columns of $\mathbf{Z}$ so that the covariance of the columns is an identity matrix. I have two cases here.

  1. When the columns are uncorrelated and identically distributed, then the $P \times P$ covariance matrix $M$ of the columns of $\mathbf{Z}$ is Toeplitz. In my problem, I can also obtain expression of the $m$th diagonal element of $\mathbf{M}$. Then, I can whiten the measurements by multiplying $\mathbf{Z}$ on the left by $\mathbf{M}^{-1/2}$.

  2. In the second case, the columns are not uncorrelated. I can still obtain the expressions of cross-correlations $R_{i_1,i_2}[j_1,j_2]$ between any two $(i_1, j_1)$th and $(i_2, j_2)$th elements of $\mathbf{Z}$. When the columns were uncorrelated, i.e. the first case, then $R_{i_1,i_2}[j_1,j_2]$ is same for any $j_1=j_2$ and zero otherwise. However, this is no longer the case now for $R_{i_1,i_2}[j_1,j_2]$. So, how can I simultaneously whiten the correlated columns $\mathbf{z_1}$, $\cdots$ $\mathbf{z_K}$?

From what I read, it looks like I should first use Principal Component Analysis (PCA) to decorrelate the columns and then apply the whitening process for uncorrelated columns as described above in the first case. However, two aspects are not clear to me:

  1. Most PCA examples I found in online resources show decorrelation of only two vectors. However, here, I want to simultaneously decorrelate $K$ correlated column vectors.

  2. If I apply PCA, then it should result in a modified expression of $R_{i_1,i_2}[j_1,j_2]$ that I can later use to construct the matrix $M$. It is not clear to me how should I be able to obtain such an expression using some PCA-related properties.

Any help would be greatly appreciated.


[Update] In the above, I have assumed the means of the elements of $\mathbf{Z}$ are zero. For non-zero means, in both correlated and uncorrelated cases, I understand that I should first element-wise subtract the respective means from matrix $\mathbf{Z}$ before moving forward with decorrelation and whitening.

  • $\begingroup$ There are several unclear or confusing points in your question. One ia about Toeplitz. Why not just diagonal cov. matrix? Another is that using PCA to decorrelate one vector (?). $\endgroup$ – ttnphns Sep 7 '17 at 10:30
  • $\begingroup$ PCA is regularly used to whiten variables. The whitened result is called standardized PC scores. $\endgroup$ – ttnphns Sep 7 '17 at 10:32
  • $\begingroup$ @ttnphns I am referring to the covariance matrix of a particular column of $\mathbf{Z}$. So $\mathbf{M} = E\{\mathbf{z_i}^T\mathbf{z_i}\}$. This is Toeplitz and is same for all columns or $\forall i$. However, if I compute the cross-covariance matrix of any two columns, then it will be diagonal because the columns themselves are uncorrelated. $\endgroup$ – ryan80 Sep 7 '17 at 16:03
  • $\begingroup$ @ttnphns Regarding your comment on PCA, I made an edit in the question. I meant to ask for a PCA example which shows decorrelating more than two random vectors. $\endgroup$ – ryan80 Sep 7 '17 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.