How to use SVD for dimensionality reduction After reading several "tutorials" on SVD I am left still wondering how to use it for dimensionality reduction.
Here is my confusion in an applied setting. If I limit svd to only considering the first two singular values / vectors and "recreate" the matrix, the dimensionality is still the same (4 columns). What should be done here to instead only use 2 columns?
data(iris)
s<-svd(iris[,-5])

u<-as.matrix(s$u[,1:2])
    v<-as.matrix(s$v[,1:2])
d<-as.matrix(diag(sing$d)[1:2, 1:2])

s2<-u%*%d%*%t(v)

 A: I think your confusion comes from the fact that the PCA truncation is going to reconstruct the full dimensions of the original matrix. If you want to only consider the first two columns of the data, then this has to be what you decompose with svd.
The first example is a truncation of the iris data using all 4 columns (as in your example) and then truncating with one PC:
dat <- as.matrix(iris[,-5])
s <- svd(dat)
plot(cumsum(s$d^2/sum(s$d^2))) # % explained variance

pc.use <- 1
recon <- s$u[,pc.use] %*% diag(s$d[pc.use], length(pc.use), length(pc.use)) %*% t(s$v[,pc.use])

x11(6,6)
par(mfcol=c(1,2), mar=c(1,1,1,1), oma=c(0,3,1,0))
zlim=range(dat, recon)
image(dat, zlim=zlim, yaxt="n", xaxt="n", ylab="", xlab="", main="Iris data")
axis(2, at=seq(0,1,,ncol(dat)), labels=colnames(dat))
image(recon, zlim=zlim, yaxt="n", xaxt="n", ylab="", xlab="", main="Truncated")

 
The second example is an svd on only the first two columns of iris, thus the reconstruction is also only going to have two columns. The reconstruction again uses the single leading PC:
dat <- as.matrix(iris[,-c(3:5)])
s <- svd(dat)
plot(cumsum(s$d^2/sum(s$d^2))) # % explained variance

pc.use <- 1
recon <- s$u[,pc.use] %*% diag(s$d[pc.use], length(pc.use), length(pc.use)) %*% t(s$v[,pc.use])

x11(6,6)
par(mfcol=c(1,2), mar=c(1,1,1,1), oma=c(0,3,1,0))
zlim=range(dat, recon)
image(dat, zlim=zlim, yaxt="n", xaxt="n", ylab="", xlab="", main="Iris data")
axis(2, at=seq(0,1,,ncol(dat)), labels=colnames(dat))
image(recon, zlim=zlim, yaxt="n", xaxt="n", ylab="", xlab="", main="Truncated")


