PCA to decorrelate variables

Question

I have 2 variables that I want to decorrelate. I was told I can use PCA to do so. I did PCA on the data and got all the parameters. Now how do I get the new set of transformed data that no longer correlates with the second variable? I want to use this set for further analysis. thanks,

Answer 1

With just two variables $X$ and $Y$, there are two sample variances $s^2$ and $t^2$, respectively, and the sample correlation coefficient, $r$. If you standardize the variables in the usual way to have unit variances, so that $\xi = X/s$ and $\eta = Y/t$, then the two principal components are

$$PC_1 = \xi+\eta = X/s + Y/t, \quad PC_2 = \xi-\eta = X/s - Y/t.$$

As a check, note that Covar($PC_1, PC_2$) = Var($X/s$) - Var($Y/t$) = $1-1=0$, proving the components are orthogonal (uncorrelated).

Visually: when you plot a scatterplot of $X$ and $Y$ in which the coordinate axes are expressed in standard units and have an aspect ratio of 1:1, then the axes of the point cloud fall along diagonal lines parallel to $X=Y$ and $X=-Y$.

In this example the variances are $s^2 = 0.98$, $t^2 = 7.90$ and the correlation is $r=-0.67$. Because $X$ and $Y$ are plotted on standardized scales with unit aspect ratio, the major axis of the cloud is diagonal (downward, due to negative correlation). This is the first principal component, $X/s-Y/t$. The minor axis of the cloud is also diagonal (upward) and forms the second principal component, $X/s+Y/t$.

Answer 2

It varies with your software, but you should have something like a component score matrix. Multiply that with your original variables to get the new set of transformed data.

Answer 3

You have loading for each component (P1, P2,....Pi).

$$P_1=l_1x_1+l_2x_2+...+l_jx_j$$ $$P_2=l_1x_1+l_2x_2+...+l_jx_j$$ $$.$$ $$.$$ $$P_i=l_1x_1+l_2x_2+...+l_jx_j$$

where $x$ is orginal data and $P_i$ is rotated component. The important is loadings ($l_1,l_2,...,l_j$). If you mulitple them with orginal data then you will get rotated Principle components. For more information check this URL.