I am not a PCA expert, nor do I have a good knowledge of linear algebra, so bear with me and my ignorance.

I am trying to understand how the authors of some papers I have been reading decorrelate two collinear variables A and B that I eventually need to use for regression analysis. In the paper, they just say decorrelate variables via PCA by orthogonalizing A against B. From what I could find on the internet, to do this you do the following:

  1. zero-center the two variables (i.e., subtract the mean of each variable from it original values)
  2. calculate the covariance matrix
  3. calculate the eigenvectors and values
  4. calculate the dot product of the original values of the two variables and the eigenvectors.

So far so good, I guess. The authors of paper report "intercorrelations" between original and decorrelated variables and report two different transformed variables, which they name A-against-B and B-against-A. I am having trouble understand what they meant by this. My thought, which could be completely off, has to do with the geometric interpretation of decorrelation as rotation. In principle, you could rotate the scatter of the two variables in either of two directions. You could make it orthogonal with the x-axis (and parallel with the y-axis) or orthogonal with y-axis (parallel with the x-axis). What I am left wondering about -- how could the direction of the rotation be mathematically determined?

Any suggestion is very much appreciated!

  • 1
    $\begingroup$ See en.wikipedia.org/wiki/Principal_component_regression $\endgroup$ – Jean-Claude Arbaut Jan 25 at 14:23
  • 1
    $\begingroup$ Hi @Jean-ClaudeArbaut, thank you for your reply. Would you mind expanding on your answer a bit? Wikipedia is very opaque for non-mathematicians... $\endgroup$ – RobertP. Jan 25 at 15:00
  • 1
    $\begingroup$ do you have a link to the papers you're talking about? $\endgroup$ – MathFoliage Jan 28 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.