I understand the basic intuition behind PCA: reducing the dimensionality of data by finding the eigenvectors along which there is most variance in the data, and projecting the data along these eigenvectors (the principal components).

What I don't understand is the following:

  • How are the eigenvectors found? A standard eigenvector equation is given by $Av=\lambda v$, where $\lambda$ and $v$ are the eigenvalues and eigenvectors respectively. So what is the $A$ matrix - the data itself, or the covariance matrix of the data... or something else? (If the data matrix isnt square then this equation doesn't hold.)

  • Once PCA has been performed / trained on a data set, can it be applied to reduce the dimensionality of new unseen data? For this to be true, I suppose a mapping would need to be output by PCA, and this mapping could be applied to the new data, say in the form of matrix multiplication.

    1. What are the outputs of PCA?
    2. How are the outputs applied to new data, if at all?
  • 2
    $\begingroup$ Q1: $A$ is the covariance matrix. Q2: you multiply your new data with $v$. $\endgroup$
    – amoeba
    Feb 7, 2018 at 9:34
  • $\begingroup$ Aren't there multiple $v$'s? How would I multiply my new data by $v$ - which $v$? $\endgroup$
    – PyRsquared
    Feb 7, 2018 at 9:38
  • 2
    $\begingroup$ Well if you want 2 PCs, then you take two eigenvectors corresponding to the two largest eigenvalues, and then multiply with those. Usually you have them stacked in a matrix V as columns, and can multiply your data directly with XV. $\endgroup$
    – amoeba
    Feb 7, 2018 at 9:41
  • $\begingroup$ stats.stackexchange.com/questions/210666/… $\endgroup$
    – Sycorax
    Feb 7, 2018 at 17:34

1 Answer 1


I will answer each question:

  • $A$ is indeed the covariance matrix (so $X^TX$ assuming $X$ is standardized)
  • The output of PCA is 3 things: the vector of column means $\mu$ of $X$, the vector of column stddevs $\sigma$ of $X$ and the rotation matrix $R = [v_1 ... v_p]$. Therefore, for a new sample row $x_0^T$, to compute its projection onto Principal Component Space, you have to standardize and rotate, that is $((x_0 - \mu) / \sigma)^T R$, which will yield a row vector with $x_0$ in PC coordinates. Please note that here I'm dividing by $\sigma$ elementwise.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.