3
$\begingroup$

I have generated 100 2D correlated MVN variables in R, on which I run prcomp. When I plot the projected points along the first principal component (in the original coordinates) with the original data overlaid, I have a bunch of points along a line (as expected) yet they do not seem to correspond to the original points in the scatterplot (I was expecting each point along the line to be the projection of some original point)

Here is my code:

require(MASS)
x <- mvrnorm(100, mu = c(0,0), Sigma = matrix(c(1,-.85,-.85,5), 2, 2),
              empirical = FALSE)  
pcX <- prcomp(x, retx = TRUE, scale = FALSE)  
transformed <- pcX$x[,1] %*% t( pcX$rotation[,1] )  
plot(transformed, col = "red")  
points(x, col = "green")  

Is there some scaling going on which is preventing me from recovering the original data, or is my understanding of PCA (or R) lacking?

$\endgroup$
  • $\begingroup$ I'm pretty new to PCA myself, but do you think it could have anything to do with the fact that there's a second component? $\endgroup$ – Atticus29 Sep 5 '13 at 2:48
  • $\begingroup$ including the second component would return the cloud of points in a different coordinate system. I want to see the 1 dimensional approximation. $\endgroup$ – Statter Sep 5 '13 at 4:53
4
$\begingroup$

Well, not a scaling, but you (implicitly) have centered = TRUE, so you need to undo the effect of centering after the rotation:

plot(x, asp = 1, col = 3)
transformed <- pcX$x %*% t(pcX$rotation)
transformed <- scale(transformed, center = -pcX$center, scale = FALSE)
points(transformed, col = 2, pch = 19, cex = 0.5)

reconstructs your original data.

Now, if you want to use only PC 1, you need to multiply score 1 (no score 2 involved!) with the inverse (for PCA = transpose) of loading 1:

plot(x, asp = 1, col = 3, pch = 19, cex = 0.5)
transformed  <- pcX$x[, 1] %*% t (pcX$rotation[1, ])
transformed <- scale(transformed, center = -pcX$center, scale = FALSE)
points(transformed, col = 2, pch = 19, cex = 0.5)
segments(x[, 1], x[, 2], transformed[, 1], transformed[, 2])

PCA projection

Note that the segments will only show an orthogonal projection if the plot has asp = 1.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The aspect ratio must be set to 1. That solved it. Nice graph $\endgroup$ – Statter Sep 5 '13 at 10:29
3
$\begingroup$

Tip #1, always do set.seed() so we can see the same random numbers as you do.

Also its possible that you aren't seeing all your original points because your plot is on the rotated data. Let's do the plot of the data points first and then add the rotated data:

set.seed(310366)
require(MASS)
x <- mvrnorm(100, mu = c(0,0), Sigma = matrix(c(1,-.85,-.85,5), 2, 2),
              empirical = FALSE)  
pcX <- prcomp(x, retx = TRUE, scale = FALSE)  
transformed <- pcX$x[,1] %*% t( pcX$rotation[,1] )  
plot(x)
points(transformed,col="red")

To see how the original data projects to the points on the line, lets use the segments function.

segs = cbind(transformed,x)
segments(segs[,1],segs[,2],segs[,3],segs[,4],col="green")

principal components

Remember you aren't fitting a linear model here, so the points don't map straight to the nearest point on the line, you're doing a rotation and then ignoring one of the axes.

If you plot the fully transformed points and the segments, you should see the rotation:

> xyt=pcX$x %*% pcX$rotation
> plot(x)
> points(xyt,col="red")
> segs = cbind(x,xyt)
> segments(segs[,1],segs[,2],segs[,3],segs[,4],col="green")

and then your plot is essentially flattening that one. I think.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ So I was wrong to assume that PCA would merge points along the perpendicular onto the principal component vector. I can't see a pattern to the rotation and flattening though. (1) Is there some intuitive explanation regarding the underlying process? (2) Is there a reason why the original points weren't recovered precisely in your second plot? $\endgroup$ – Statter Sep 5 '13 at 7:17
  • $\begingroup$ Ah ha. You've ignored the centering that prcomp does when you've computed your coordinates. Try adding center=FALSE to your prcomp call and then repeating the segment plot. $\endgroup$ – Spacedman Sep 5 '13 at 8:03
0
$\begingroup$

I think there were some errors in the reply by cbeleites: I just corrected them. Be careful with this, in this example, it happened that we only have 2x2 matrix and it is symmetric.

plot (x, asp = 1, col = 3, pch = 19, cex = 0.5)
transformed  <- pcX$x [,1] %*% t (pcX$rotation **[,1]**
transformed <- scale (transformed**)**, center = -pcX$center, scale = FALSE)
points (transformed, col = 2, pch = 19, cex = 0.5)
segments (x [,1],x [,2], transformed [,1], transformed [,2])
| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Because your code is (a) unexplained and (b) not even syntactically correct, it is difficult to ascertain what corrections you are trying to make. $\endgroup$ – whuber Mar 31 '16 at 21:49
  • $\begingroup$ You should read the previous post before writing $\endgroup$ – Yucheng Song Apr 7 '16 at 20:48
  • $\begingroup$ Since your code obviously doesn't execute and you don't explain what errors it is intended to fix, nor how it answers the original question, I don't see how it helps. I have to skip back and forth between cbeleites' answer and yours just to figure out what you're trying to say. Surely you can clarify this post to make it possible to understand without so much effort on the part of your readers. $\endgroup$ – whuber Apr 7 '16 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.