# Does PCA do something else apart from selecting features with the most variance?

While experimenting with Spark library MlLib, I questioned myself if I understood well the mechanism of PCA algorithm, because output of MlLib algorithm was not what I expected to get. so for given data of 20 features and 10 observations:

val rdd: RDD[Vector] = sc.range(0, 20).map { row =>
val values = 1.to(10).map { _ => rand.nextInt(2).toDouble }
Vectors.dense(values.toArray) }


I applied pca

val pca = new PCA(5).fit(rdd)


and then transform my initial data :

  val finalRdd = rdd.map { vector =>
val newVector = pca.transform(vector)
println("Before: " + vector)
println("After : " + newVector)
newVector }


I expected to have the rdd only with 5 features, but the values of those features should be the same, as in the initial rdd. As far as I understand pca, it keeps only the features/vectors that have the most important variance score. But instead of this I've got the rdd of 5 features, but values of those features were modified in the way I don't understand.

Before: [1.0,1.0,1.0,1.0,0.0,0.0,0.0,0.0,1.0,0.0]

After : [1.14461218805706,0.8986586176492057,-0.580979228415323,0.09346270935831252,0.238925870721517]

Before: [1.0,1.0,0.0,1.0,0.0,0.0,0.0,1.0,1.0,0.0]

After : [1.1606629979632594,0.6110896934565042,-0.3717757311830274,-0.22954508416826663,0.18370214647002148]

Before: [0.0,1.0,0.0,1.0,1.0,1.0,0.0,1.0,1.0,0.0]

After : [1.6723486323814756,0.41082473274832476,-0.7526031076025419,-0.26291312583276166,0.11992366811728228]

Could you explain me why those features are modified? and when my understand of pca is wrong? Does the algorithm make something else with the data, and not only selection of features?

• PCA itself is a (dimensionality reducing) feature construction, not feature selection, method. Jun 5, 2016 at 20:39

As for feature selection, what is usually done is to utilize the fact that the eigenvectors are ordered in decreasing variance, which is quantized by the eigenvalues. So when you select the first K (which is <N or <<N) components, you still hold the axes that explain a lot (90% or 99%) of the variance. Therefore, instead of your original N dimensions, you can continue your task with a smaller set of K dimensions (combinations of input features) without much information loss.
• As for feature selection... I'm sorry. What you are describing in that paragraph is not feature selection. The latter implies selecting a subset of intact features, and not their linear combinations of input features. Jun 5, 2016 at 20:43