Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm trying to estimate a correlation matrix from the 5 largest eigenvalues and associated eigenvectors of the sample correlation matrix. My problem is that the output from the following Matlab code results in much higher average correlation (0.8 v 0.4) for the financial data I'm looking at.

[V,D] = eig(InputCorr);
eigvals = diag(D);
eigvals(1:(end-5)) = 0;
eigvals = eigvals*size(InputCorr,1)/sum(eigvals);
BB = bsxfun(@times,V,eigvals')* V';
T = 1 ./ sqrt(diag(BB));
OutputCorr = 0.5*(BB+BB') .* (T*T'); 

Is such a marked increase in average correlation to be expected or am I doing something wrong?

Update: The $A=λ_{1}v_{1}v_{1}' + ... + λ_{n}v_{n}v_{n}'$ decomposition suggested by @Jonathan Lisic is very useful. If I truncate it to just the first five terms, then the off-diagonal elements of the matrix are what I expect (similar to the corresponding elements of the original matrix and similar average correlation), but the diagonal elements are well below 1. So the increase in correlation is caused by rescaling the matrix to get the diagonal elements to equal to one. I am actually only interested in the off-diagonal elements, is it kosher to use these directly without doing any rescaling?

share|improve this question
1  
I've tried your code with correlation matrix of random normals, and there were no marked increase in average correlation. So probably the problem is as Jonathan explained, the eigenvalues you discard carry substantial information about your input matrix. I have few questions about your code though. Why do you do the transformation in 4 line, i.e. inflate eigenvalues? Why do you do tranformation 0.5*(BB+BB') in the last line? BB is symmetric, so BB=0.5*(BB+BB'). –  mpiktas Sep 22 '11 at 8:03
    
The inflation of the eigenvalues was to ensure that the sum of the eigenvalues equals the trace of the correlation matrix. The 0.5*(BB+BB') is to enforce symmetry, I came across cases where there were numerical issues (due to use of bsxfun, which is much faster) which caused problems in subsequent analysis. –  MatlabSorter Sep 22 '11 at 12:23

2 Answers 2

This is a quick example in R for taking a 5x5 positive definite matrix and turning it into an approximation of the first matrix.

#Test Matrix
A <- matrix(1,nrow=5,ncol=5)
diag(A) <- 1:5

evec <- eigen(A)$vectors
eval <- eigen(A)$values

# we can get the original matrix back through
# spectral decomposition
evec %*% diag(eval) %*% t(evec)

# since R and most other software automatically sorts
# the eigenvalue/vector pairs by absolute magnitude we
# can just do the following subsets to approximate A

evec[,1:3] %*% diag(eval)[1:3,1:3] %*% t(evec[,1:3])

The difference you are seeing might be from your code. In this case I would check if you could generalize your function to reconstruct the matrix for any subset of the eigenvalues/eigenvector pairs and test it for the case where you include all the pairs. This case should match the original matrix.

The other cause for the large change is that the sum of the deleted eigenvalues is reasonably large. Recall from the spectral decomposition of a positive definite matrix we can write the matrix as follows $A = \lambda_1 v_1 v_1^T + ... + \lambda_n v_n v_n^T $. So by subsetting the deleted values and reconstructing this portion of A, you will get what was lost by your approximation.

share|improve this answer
    
If I keep the full set of eigenvalues/eigenvectors I do recover the original matrix. Keeping just a few of the largest eigenvalues will definitely lose some information, but I think the gap in my understanding is why this would cause such an increase in correlation. I would expect that discarding information about the correlation structure would push the average correlation towards zero. –  MatlabSorter Sep 22 '11 at 12:23
    
Consider each eigenvalue/vector pair as approximate covariance matrices $ \lambda_i e_i e_i^T $ Then the correlation in this matrix is actually the correlation between the eigenvector and the other original values, these may be negatively or positively correlated, so if you remove an eigenvalue that is negatively correlated with an original variable, you can cause an 'increase' in the correlation between different original variables. Although the maximum change in any direction for a covariance matrix element is $ \lambda_i $ per eigenvalue/pair removal. –  Jonathan Lisic Sep 22 '11 at 20:21

I can't follow your code because I'm not Matlab user, so I have difficulty to discern whether you are trying to restore your correlations via eigenvectors or loadings. According to basic factor theorem, restored R matrix = A*A', where A is the matrix of component (or factor) loadings. Loadings are eigenvectors normalized by corresponding eigenvalues, so that column sum-of-squares of A are the eigenvalues.

Now, the major difference between Principal Component Analysis (PCA) and Factor Analysis proper (FA) is that PCA restores R matrix only by all p components: A should be square. FA, if good "trained", can restore R with just m (m lesser than p) factors: A may be rectangular. As soon as you are performing PCA rather than FA there's no wonder that you fail to restore correlations by only m (=5) components.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.