4
$\begingroup$

For $z=W^\mathbf{\intercal}(x-m)$ where $k$ columns of $W$ are the top $k$ principal components of a given dataset $x$ and $m$ is the mean, how can I prove that $z$ has diagonal covariance matrix?

Here $x$ is the original dataset ($d \times N$ dimensional matrix) and $m$ is the mean of $x$ ($1\times N$ dimensional matrix) and $k$ is a number less than $d$.

$\endgroup$
2
  • 1
    $\begingroup$ I assume $X$ and $M$ are matrices? Perhaps a $k \times N$ matrix? $\endgroup$
    – ahwillia
    Jan 9 '14 at 15:35
  • $\begingroup$ This is a very sloppily formulated question; e.g. the expression $x-m$ doesn't really make sense as these are two matrices of different sizes. I edited to make it at least a bit less sloppy. $\endgroup$
    – amoeba
    Jan 13 '15 at 12:20
6
$\begingroup$

I think the columns of $W$ are eigenvectors of the covariance matrix.

$(x-m) \cdot (x-m)^T \cdot (w_1, w_2, ... w_k) = (\lambda_1w_1, \lambda_2w_2,...\lambda_kw_k)$

so,

$(w_1, w_2, ... w_k)^T \cdot (x-m) \cdot (x-m)^T \cdot (w_1, w_2, ... w_k)$

$= (w_1, w_2, ... w_k)^T \cdot(\lambda_1w_1, \lambda_2w_2,...\lambda_kw_k)$

$= diag[\lambda_1,\lambda_2,...\lambda_k]$

Since

$(x-m)^T \cdot (w_1, w_2, ... w_k) = ((w_1, w_2, ... w_k)^T \cdot (x-m))^T$,

$z = diag[\sqrt\lambda_1,\sqrt\lambda_2,...\sqrt\lambda_k]$

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.