# First principal component does not correlate with its variables

I am building an indicator for developable land as the inverse of the principal component of four variables that are all negatively correlated with land availability (e.g. terrain slope, fraction of soil covered by water, etc.).

This is the output of my analysis

Principal components/correlation                 Number of obs    =     41,278
Number of comp.  =          4
Trace            =          4
Rotation: (unrotated = principal)            Rho              =     1.0000

--------------------------------------------------------------------------
Component |   Eigenvalue   Difference         Proportion   Cumulative
-------------+------------------------------------------------------------
Comp1 |      1.78414      .741535             0.4460       0.4460
Comp2 |       1.0426      .122243             0.2607       0.7067
Comp3 |      .920361      .667465             0.2301       0.9368
Comp4 |      .252896            .             0.0632       1.0000
--------------------------------------------------------------------------

Principal components (eigenvectors)

--------------------------------------------------------------------
Variable |    Comp1     Comp2     Comp3     Comp4 | Unexplained
-------------+----------------------------------------+-------------
land_use_p~6 |  -0.6829    0.0519    0.2148    0.6962 |           0
water_fra~06 |  -0.0330    0.8235   -0.5609    0.0792 |           0
edge_dens~06 |   0.6936   -0.0920   -0.0755    0.7105 |           0
avg_slope |   0.2268    0.5575    0.7960   -0.0647 |           0
--------------------------------------------------------------------


I then keep the first component and take its inverse and these are the correlations between the original variables and 1/pc1

             |  pc1_inv lan~c_06 wat~n_06 edge_~06 avg_sl~e
-------------+---------------------------------------------
pc1_inv |   1.0000
land_use_p~6 |  -0.0042   1.0000
water_fra~06 |  -0.0088  -0.0121   1.0000
edge_dens~06 |   0.0082  -0.7399  -0.0666   1.0000
avg_slope |   0.0126  -0.1002   0.0531   0.1603   1.0000


They are all pretty low correlations and two of them are positive when they are supposed to be negative.

It is the first time I use PCA for my analysis so I might be missing something obvious about it. Any ideas of why this could be the case?

Also, correlation between the principal component and its variables

             |      pc1 lan~c_06 wat~n_06 edge_~06 avg_sl~e
-------------+---------------------------------------------
pc1 |   1.0000
land_use_p~6 |  -0.9119   1.0000
water_fra~06 |  -0.0319  -0.0169   1.0000
edge_dens~06 |   0.9253  -0.7410  -0.0626   1.0000
avg_slope |   0.3164  -0.1081   0.0547   0.1651   1.0000

• What are the correlations between the first component, not inverted, and the original variables? Commented Oct 17, 2022 at 19:31
• Good point. I'll add them above. Commented Oct 17, 2022 at 19:34
• Here is a Python gist performing a similar analysis computing the reciprocal of the first principal component. Commented Oct 18, 2022 at 3:05

If you look at the correlations between the first component and the original variables, do you see a problem? I don't! Therefore, we can deduce that the problem was caused by inverting the component. Why might that be?

Note what inversion does - large negative numbers become small negative numbers, and large positive numbers become small positive numbers. So, large negative values for the principal component are moved to be "next to" large positive values for the principal component, and they are both made into small values. Simultaneously, small negative numbers become large (relatively speaking) negative numbers, and likewise for small positive numbers. These numbers, which were in some sense "next to" each other, are moved farther away from each other and are made larger in absolute value than the formerly-large values.

The first one of these two effects - switching magnitude - will act to break correlations. The second one will act to reverse their sign. Both of these are effects you've observed with your data.

Here's an example in R:

x <- -10 + 20*runif(1000)
cor(x,x)
[1] 1
cor(x,1/x)
[1] 0.02741836

• Here is a Python gist for the same type of simulation example. Commented Oct 18, 2022 at 2:55
• Thanks! This solves part of the issue. The other one is that the signs of the correlations do not all go in the direction I expect them to go. Is that a data issue? Commented Oct 18, 2022 at 14:35
• Why would you expect them to go in any particular direction? It's not like you've built a model of available land as a function of the four variables; the principal component just looks at (linear) relationships between the four variables themselves. Try printing out a correlation matrix between them; I'm guessing you will see that land_use and edge_dens are strongly negatively correlated, for example. Commented Oct 18, 2022 at 14:48
• I am basically replicating and extending what another paper does and their claim is that since all the components correlate negatively with pc1, then pc1 is a good index for land availability. Is the claim incorrect? Commented Oct 18, 2022 at 19:27
• Well, yes. It may or may not be a good index for land availability, but the logic, as stated, isn't valid. Is there no variable for "land availability"... available? If there were, you could model land availability as a function of the other variables; without it, all anyone has is speculation. Commented Oct 18, 2022 at 20:23