PCA to choose variables based on its loadings on PC1 [duplicate]

Question

I have a dataset of cave dimensions (and other variables related to their features).

The problem is that 3 of these variables are: Length, Area, and Volume. These 3 are highly correlated as they basically are: Area = Length * Length and Volume = Area * Length.

I know using PCA I can find the variables which influence the most the whole set's variation.

Is that ok to perform PCA and pick only the most 'influential' variable to my further analysis? Say, Area for example (I think Area is the best one, not sure, just a guess).

I know PCA does not choose variables. Instead, it creates new ones. However, I know the PC1 (being the major 'influencer' on the variation) is formed by loadings of the previous variables.

Does it make sense to pick then the variable which contributes with the most loadings on PC1?

Answer 1

The question and some other answers seem to assume or imply that caves have simple shapes such that length, area and volume are broadly equivalent choices that just happen to have different dimensions. I don't have the data but on general geological (geomorphological) grounds I doubt that is the case.

Caves in practice are likely to be much more irregular in shape than say bricks or eggs or other volumes. But even if it were true as a first approximation that say

Area $= b$ Length$^2$

or

Volume $= c$ Length $\times$ Area

these are multiplicative relationships that won't play well with a correlation and PCA framework. You would almost certainly be much better off working with log Length, log Area, log Volume. Then your first approximations don't feature squares or cubes as powers. Log scales would also, I guess, work much better with what are likely to be highly skewed distributions with outliers present too. (Most caves are small, but a few are enormous.)

Note that you really don't need PCA to know what works best here. You can and perhaps should just look directly at correlations and scatter plots.

Posting (example) data would allow illustration. It seems that your main goal is using size variables as predictors, so telling us more about what you want to do would be a good idea.

There is a broader point here that statistical analysis should be informed by dimensional analysis when it applies. For details of a splendid article on this by David Finney (1917$-$2018) see https://www.jstor.org/stable/2346969

Answer 2

PCA doesn't do variable selection. Instead, it does dimensionality reduction, which isn't the same. In your example, PCA might tell you that actually $3L + 5A - 2V$ explains a lot of the variation in your data, but no other combinations of $L, A, V$ do anything useful whatsoever.
This may of course solve the problem you have, but it doesn't tell you to that only one of the variables is useful.

As a side note, having variables which are correlated in this way isn't necessarily a bad thing and can in fact be a good thing.
If you want to do linear regression with your features, then you mostly need to be concerned about variables for which there is a linear relationship. But in your case there is a polynomial relationship between the variables, so you should be fine.
Further, polynomial regression is often very useful if the original relationship wasn't really linear. In your case, using $L, A, V$ as variables would roughly correspond to doing a polynomial regression with $L, L^2, L^3$.
To get an idea which of $L, A, V$ would be a good variable to include, you could look at a scatterplot of $L, Y$ and look whether it looks linear, quadratic or cubic.

Answer 3

I think the variables that in this case explicate the most variance could be Length, since ideally you should be able to represent all your features in relationship to this one.

Furthermore PCA just describes the data as a new set of variables,or axes, (that may not be your initial one) that maximize the variance in your dataset. So I think in this case you could just keep the variable "Length" since the variance of the others it is just a derivation of the variance on the variable "Length".