How is the principal component applied on to the data?

I am bit unsure (Or might be overthinking this) or is the chosen PC somehow applied on to the data, to reduce the dimentionality of the data, or how does one use PC to do any form of mathematical computation?...

I mean each principal component is a linear combinations of the variables.. It basically is an eigenvector and eigenvalue stating the pattern of the data.

Most textbooks seem to end the elaboration after having found the eigenvectors and the eigenvalues, but aren't there another step which implies reducing the data with the PC found?...

• Most textbooks seem to end the elaboration... Really? Do they ever mention the computation the values of the first m PCs, thereby reducing the dimensionality? Commented Jun 19, 2016 at 13:17
• I get how they compute PC's and so on... but aren't the components somehow applied to the original dataset.. each PC is just a linear combinations.. I know that they reduce the dimentionality, but not sure how a PC can reduce the dimentionality of the dataset, as they aren't applied on to the original dataset , only extracted from it. Commented Jun 19, 2016 at 13:19
• Principal components can be used to restore the original variables data. Commented Jun 19, 2016 at 13:21
• If i used all I would have the complete dataset, thats right, what if you only wanted to have 50% variance. Well one way would be to choose m numbers of PC's such that the requirement is fulfilled, and then what? Commented Jun 19, 2016 at 13:24
• Principal components don't reduce dimensionality: if dataset has $n$ features, we can compute $n$ PCs, so dimensionality doesn't change. Dimensionality reduction takes place when we use an arbitrary number of PCs instead of original features in dataset. Commented Jun 19, 2016 at 15:55

Given you have found the principal modes of variation in your sample (eigenvectors) you use these as your new axis system. When you compute the PC scores you simple project the data on the axial system defined by these eigenvectors.

Let's say your original data is an $N \times p$ matrix $X_0$ with a valid covariance $p \times p$ matrix $C$ (the columns of $X_0$ are assumed to have mean $0$). The covariance $C$ can then be eigen-decomposed as $C = U \Lambda U^T$ where $U$ is the $p \times p$ matrix of eigenvectors (each column is an eigenvector) and $\Lambda$ is the diagonal matrix holding the corresponding eigenvalues. This is the most important step, now $U$ can allow a change of basis from $X_0$ to $Z$, where we define $Z = X_0 U$, $Z$ are now the projected scores $N \times p$ matrix; these are the projections of the original data $X_0$ onto the axis defined by the columns of $U$. Note that here we used the full matrix $U$. If we used the optimal $k$-th dimensional approximation of the data $X_0$ we would use only the first $k$ columns of $U$. As you see the data $X$ are directly employed for their dimensional reduction using the eigenvectors $U$. (Sometimes the columns of $U$ are also called loadings.)

• Isn't there something wrong.. should it not be $Z = U X_0^T$ Commented Jun 22, 2016 at 19:08