This is a follow-up to the question asked regarding SVD and dimensionality reduction (question).
In that question I asked how to use SVD for dimensionality reduction. Although not stated, the ultimate goal here is to use the reduced feature set and input them into a classification or regression algorithm.
I have learned that SVD is a technique used by
prcomp in R, as the "v" matrix from a run of svd on a centered and scaled matrix is the same as the loadings (eigen vectors) from a PCA using the traditional eigen decomposition on a correlation matrix:
data(iris) #these two match eigen(cor(iris[,-5])) #eigen vectors svd(scale(iris[,-5]))$v
This has helped with my understanding of the connection between SVD and PCA. However, I have two additional questions:
1) Why do the following differ in signs for the first PC? Is this OK?
2) To match the output of prcomp, one can multiply the scaled/centered original data by the 'v' matrix from SVD:
PCSCORE1<-scale(iris[,-5]) %*% SVD2$v[,1:2] #PC scores from SVD PCSCORE1[1:10,] #PC scores from first 2 PC #matches this PCA<-prcomp(iris[,-5], center = TRUE, scale =TRUE) PCA$x[1:10,1:2]
but I have seen in multiple locations (e.g. question) and the rapidminer package (a machine learning tool written in JAVA) that just the 'u' matrix that results from running svd on the center/scaled input matrix X is used as the PC scores.
What is the connection of u to Xv and if u can be used, why does prcomp compute Xv? Mechanically u is Xvdiag(1/d) so the eigen vectors related to the largest eigen values are scaled down - why is this used?