0
$\begingroup$

did I understand pca correctly?

if we have 9 features we call PCA to find corresponding components, then for example we use 2 first PCA, now we have a data set with 2 variables? and they are linear combination of our 9 features? now regression can be done on this 2 variables rather than 9?

$\endgroup$
0
$\begingroup$

If you have 9 features, then your data is 9 dimensional. These directions are like $x,y,z$ (as in 3D), but there are 9 of them in total. PCA finds the directions with the largest variances, i.e. your data varies along those directions the most. These directions will be linear combinations of your original dimensions. For instance, the direction of $x+2y$ (i.e. your principal component (PC) will be normalised version of this). If you take the first $k$ principal components, e.g. $k=2$ in your example, you'll have $k$ directions that the data variance is explained most. You'll project your original samples to these PCs and obtain $k$ coordinates (or called as component scores).

For example, projecting a 2D data point $[1,2]$ to $x$ direction would give you $1$, and to $y$ direction would give you $2$. These are your component scores. If you project this point to the direction of one of your PCs, whatever it is, you'll obtain a scalar as here. When you project onto $k$ of them, you'll obtain $k$ scalars.

As you can see, these $k$ new coordinates are indeed linear combinations of your previous features due to the projection operation. For instance, if one of your PCs is $v$, and let a data point be $x$, the new coordinate corresponding to it is $v^Tx=\sum_{i=1}^9 v_ix_i$, i.e. a linear combination of your previous features. You can now regress/classify using these new features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.