0
$\begingroup$

I just wanted to do this small experiment to make sure I understand PCA correctly. My dataset contains 8 columns. The first two columns are randomly generated in excel => randbetween(4, 5) and the other 6 columns are also generated in the same way but the formula used is => randbetween(1,3)

When I do PCA on this I am not getting good results. I expect that the result should indicate high eigenvalues for a factor that is a combination of first two columns and low on other colums. This is my code in R :

sensex.dat = read.csv('C:/Study/_SEM4/brand man/emperical/dice.csv', header = T)
attach(sensex.dat)
sensex.cov = cov(sensex.dat)
sensex.eigen = eigen(sensex.cov, symmetric = T)
sensex.eigen$values
sensex.eigen$vectors
| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Can you explain why you expected that result? That is probably the most important information if you want to know whether you've understood PCA correctly. :) $\endgroup$ – MånsT Aug 31 '12 at 5:39
  • $\begingroup$ The first 2 factors behave similarly and differently from the others, so shouldn't PCA combine the first 2 factors as one factor? $\endgroup$ – Prakhar Aug 31 '12 at 5:57
  • 3
    $\begingroup$ No, because 'behave similarly' means 'be Pearson correlated' for PCA. Amplitude does not really matter. $\endgroup$ – user88 Aug 31 '12 at 6:11
  • 1
    $\begingroup$ For a working R example of how to construct a random dataset with specific PCA output and how to compare the actual output to the intended output, please see the answer at stats.stackexchange.com/a/35035. $\endgroup$ – whuber♦ Aug 31 '12 at 13:38
2
$\begingroup$

As others have told you PCA does not look for amplitude - in fact it is standard procedure to normalize your variables before a PCA. You did not do this by the way. It looks for correlations between the columns.

The result you want to generate you would get by

  1. Randomly generating a column
  2. Generating a second random column with similar parameters but also adding the first column to it. In your example this would basically be first column + randbetween.
  3. Generate additional uncorrelated columns as in 1
  4. Normalize and then get eigenvalues and vectors
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ My understanding is that you wouldn't always standardize before doing PCA (you always subtract the mean however). If, for example you were trying to model salaries of different occupations and you cared primarily about the \$ accuracy of approximating with a reduced set of components you might not standardize, choosing to accept higher relative errors on low salaries. OTOH if the variables are on different scales you'd always standardize. Discussed e.g. here in more detail. $\endgroup$ – TooTone Mar 18 '14 at 21:27
  • $\begingroup$ Standardization when performing PCA on the correlation matrix (the usual approach, outside of a few fields, like morphometrics, where they use the covariance matrix) is irrelevant: cor$(\mathbf{X})$ = cor$((a\mathbf{X})+b)$ for $0 < a < \infty$, and $-\infty < b < \infty$. Covariance matrix applications of PCA entail more than simply standardizing variables, since the assumption about what each component contributes to total variance is substantively different. $\endgroup$ – Alexis Apr 24 '14 at 14:36
0
$\begingroup$

The post referred to by whuber is quite useful for artificial data. Here's a simple PCA with your random data generated in R. The first two principal components explain about 40% of the variation, the remaining six explain the rest. 

n=100
sensex.dat = matrix(NA,nrow=n,ncol=8)
sensex.dat[,1:2] = runif(n*2,4,5)
sensex.dat[,3:8] = runif(n*6,1,3)

p = princomp(sensex.dat,scale.=FALSE)
summary(p)
biplot(p,xlabs=c(rep('+',n)))
screeplot(p)

enter image description here enter image description here

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.