In both Wikipedia and this medium post, I see the succinct principle components decomposition of X represented as


However, it seems to me that it should be $T = WX$ instead, if according to the Wikipedia page that columns of $W$ are the eigenvectors of $X^TX$ and $X$ is arranged as row vectors for each observation. In short, $X \in \mathbb{R}^{nxk}$, where $n$ is the number of observations and $k$ is the dimension of each data point, and $W \in \mathbb{R}^{nxn}$, so $XW$ does not even exist.

Am I getting the derivation somewhere wrong?

  • $\begingroup$ It depends on the orientation of you data matrix $\endgroup$ – ReneBt Aug 6 at 11:27

Since the data samples have dimension $k$, the scatter matrix, $X^TX$ will be of dimension $k\times k$ leading to $k$ dimensional eigenvectors. So, $W$ is of dimension $k\times k$. With this correction, $T=XW$ indeed exists. You can also think of it as $$T=\underbrace{\begin{bmatrix}x_1^T\\x_2^T\\\vdots\\x_n^T\end{bmatrix}}_{X^T}\underbrace{\begin{bmatrix}w_1&\cdots&w_k\end{bmatrix}}_{W}=\begin{bmatrix}x_1^Tw_1&\cdots &x_1^Tw_k\\\vdots&\ddots&\vdots\\x_n^Tw_1&\cdots&x_n^Tw_k\end{bmatrix}$$ where $i$-th row of $T$ represents the projection of $i$-th data sample to the first $k$ (whatever it is) PCs.

  • $\begingroup$ Thanks! But could it be that the $x_1$ and $x_k$ in your answer are actually $w_1$ and $w_k$? $\endgroup$ – kumom Aug 6 at 12:31
  • $\begingroup$ Yes, they are. I've edited. $\endgroup$ – gunes Aug 6 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.