Does PCA do something else apart from selecting features with the most variance?

Question

While experimenting with Spark library MlLib, I questioned myself if I understood well the mechanism of PCA algorithm, because output of MlLib algorithm was not what I expected to get. so for given data of 20 features and 10 observations:

val rdd: RDD[Vector] = sc.range(0, 20).map { row =>
val values = 1.to(10).map { _ => rand.nextInt(2).toDouble }
Vectors.dense(values.toArray) }

I applied pca

val pca = new PCA(5).fit(rdd)

and then transform my initial data :

  val finalRdd = rdd.map { vector =>
    val newVector = pca.transform(vector)
    println("Before: " + vector)
    println("After : " + newVector)
    newVector }

I expected to have the rdd only with 5 features, but the values of those features should be the same, as in the initial rdd. As far as I understand pca, it keeps only the features/vectors that have the most important variance score. But instead of this I've got the rdd of 5 features, but values of those features were modified in the way I don't understand.

Before: [1.0,1.0,1.0,1.0,0.0,0.0,0.0,0.0,1.0,0.0]

After : [1.14461218805706,0.8986586176492057,-0.580979228415323,0.09346270935831252,0.238925870721517]

Before: [1.0,1.0,0.0,1.0,0.0,0.0,0.0,1.0,1.0,0.0]

After : [1.1606629979632594,0.6110896934565042,-0.3717757311830274,-0.22954508416826663,0.18370214647002148]

Before: [0.0,1.0,0.0,1.0,1.0,1.0,0.0,1.0,1.0,0.0]

After : [1.6723486323814756,0.41082473274832476,-0.7526031076025419,-0.26291312583276166,0.11992366811728228]

Could you explain me why those features are modified? and when my understand of pca is wrong? Does the algorithm make something else with the data, and not only selection of features?

Answer 1

PCA is not a feature selection method, but it can be used for that purpose.

PCA is simply a feature transformation, where it first chooses linear subspaces that are orthogonal to each other (eigenvectors), then projects your original data on these subspaces and returns you the projected vectors. That's why you see different values.

As for feature selection, what is usually done is to utilize the fact that the eigenvectors are ordered in decreasing variance, which is quantized by the eigenvalues. So when you select the first K (which is <N or <<N) components, you still hold the axes that explain a lot (90% or 99%) of the variance. Therefore, instead of your original N dimensions, you can continue your task with a smaller set of K dimensions (combinations of input features) without much information loss.