1
$\begingroup$

When one computes the vector of scores (t1) using the principal component (p1) the data is being projected over the direction of biggest variation. One could measure the distance between the point where the data was projected and the origin.

If we do the squared sum of squares of those distances (because they could be negative), we won't obtain the eigenvalue of the eigenvector pointing in the direction of the principal component. Why?

An example, as requested:

  1. The original data:

     sample = [[1.343730519  , -.160152268  ,  .186470243], 
              [-.160152268  ,  .619205620 ,  -.126684273], 
              [.186470243  , -.126684273  , 1.485549631]] )
    
  2. Eigenstuff (from the covariance matrix):

     evalues = [2.22044605e-16, 1.67438287, 2.82561713]
    
     evectors.T = [ 0.54061848,  0.65888106,  0.52307496],
                [ 0.68485977,  0.0164023 , -0.72849026],
                [ 0.48856807, -0.75206829,  0.44237374]]
    
  3. Score using just the first component (3rd vector from above)

     t1 =  [1.0619562 , -1.93803314,  0.87607695]
    
  4. The following is the part that I don't get

The elements in vector t1 are the 'distances' from the origin to the point where the original data was projected in the direction of p1 Since the eigenvalue of p1 is the magnitude of the variance in that direction, I would expect that the sum of squares of the elements in t1 would yield the same result as the plain eigenvalue. Which is not the case, computing the squared sum of squares (SS) over t1 yields

     SS = 2.3772324776675657

The eigenvalue was:

     evalue_p1 = 2.82561713

It is very similar yet not the same, why?

$\endgroup$
2
  • $\begingroup$ Please show a numerical example for it. $\endgroup$
    – ttnphns
    Sep 10 '20 at 15:32
  • $\begingroup$ Could you take a look? @ttnphns $\endgroup$
    – Ralphns
    Sep 11 '20 at 20:35
0
$\begingroup$

It seems your numerical example is off? I got a different set of eigenvectors from the covariance matrix than your example:

sample <- matrix(c(1.343730519,  -.160152268,   .186470243, 
                   -.160152268,   .619205620,  -.126684273, 
                    .186470243,  -.126684273,  1.485549631),
                 nrow = 3, ncol = 3)
eig <- eigen(cov(sample))

Output:

eigen() decomposition
$values
[1] 8.197737e-01 7.253769e-01 2.220446e-16

$vectors
           [,1]       [,2]       [,3]
[1,]  0.1112879  0.9170471 -0.3829356
[2,] -0.3858842 -0.3152184 -0.8670241
[3,]  0.9158102 -0.2442581 -0.3187939

Project the original dataset to the first principal component and compute the variance:

var(sample %*% eig$vectors[, 1])

Output:

A matrix: 1 × 1 of type dbl
0.8197737

which is consistent with the first eigenvalue.

$\endgroup$
4
  • $\begingroup$ Thanks for the answer. Did you scale the original matrix? I computed the decomposition without scaling it and got the same result as you. Perhaps the scaling has to do with the SSS being different than the eigenvalue of the cov matrix? $\endgroup$
    – Ralphns
    Sep 12 '20 at 7:16
  • $\begingroup$ Hey, just figured out what was happening, instead of just computing the SSS over the scores vector, one should compute sum of squares over (n-1) to get the eigenvalues. But again my question is the same, why the squared sum of squares is not the same as the variance in this setting? $\endgroup$
    – Ralphns
    Sep 12 '20 at 7:37
  • $\begingroup$ @Ralphns What did you mean by "compute sum of squares over (n-1) to get the eigenvalues"? Do you have an example for the calculation? $\endgroup$ Sep 12 '20 at 12:07
  • $\begingroup$ @Ralphns scaling in general will have an impact on the resultant variance, so I wouldn't be surprised the variance of the projected scores doesn't add up to the eigenvalue. $\endgroup$ Sep 12 '20 at 12:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.