0
$\begingroup$

I checked my PCA code many times, and I can't find anything wrong. I do PCA with eigen-decomposition, by using the eigen function in R. My data is 2-dimensional, I want to reduce it to one dimension. I center my data. I don't understand why the distances in the image I send don't match relatively. I even compared my solution to the results from R's PCA function prcomp and they match. I send the code and images. enter image description here

enter image description here

Edit after answer was accepted to this question, new tested working code:

enter image description here

$\endgroup$
5
  • 2
    $\begingroup$ Did you try normalizing your variables to have unit standard deviation before applying PCA? Like (data_mat[, 1] - mean(x)) / sd(x) $\endgroup$
    – mme
    May 31 '17 at 4:25
  • $\begingroup$ nope, didn't do that, I thought that doesn't matter because my original variables are randomly generated x and y values, so in some sense they can be treated as if they have the same unit i guess ... btw, at what point do you suggest i do normalization? at the beginning before computing the covariance matrix? If I do so, then when projecting, I should use that transformed data also? $\endgroup$
    – user162200
    May 31 '17 at 4:29
  • 1
    $\begingroup$ I didn't know they are random because I don't see any code for generating data_mat. If they already have unit sd and mean zero, don't worry about it. But from your left image, it appears that the x variable is not centered. The cov function in R already centers your data (otherwise it would be second moment matrix), but it doesn't normalize by SD. Try performing pca on the correlation matrix. $\endgroup$
    – mme
    May 31 '17 at 20:14
  • $\begingroup$ oh my god, I literally tried hundreds of things and read so much these days. You won't believe what turned out to be the problem in the end ! You are right about the left image. I was plotting the original uncentered/unscaled data !!! Now everything looks perfect. Thank you so so so much !!! Please send your comment as answer so I can accept it, people can still benefit from this post I think. $\endgroup$
    – user162200
    May 31 '17 at 20:33
  • 1
    $\begingroup$ Sure, glad I could help. $\endgroup$
    – mme
    May 31 '17 at 20:34
1
$\begingroup$

Did you try normalizing your variables to have unit standard deviation before applying PCA? Like (data_mat[, 1] - mean(x)) / sd(x)

I don't know if your data is random because I don't see any code for generating data_mat. If they already have unit sd and mean zero, don't worry about it. But from your left image, it appears that the x variable is not centered. The covfunction in R already centers your data (otherwise it would be second moment matrix), but it doesn't normalize by SD. Try performing pca on the correlation matrix.

$\endgroup$
1
  • 1
    $\begingroup$ I will copy-paste my comment from above here, so people can see on the answer post what the problem and solution was: "oh my god, I literally tried hundreds of things and read so much these days. You won't believe what turned out to be the problem in the end ! You are right about the left image. I was plotting the original uncentered/unscaled data !!! Now everything looks perfect. Thank you so so so much !!! Please send your comment as answer so I can accept it, people can still benefit from this post I think." $\endgroup$
    – user162200
    May 31 '17 at 20:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy