# PCA to choose variables based on its loadings on PC1 [duplicate]

I have a dataset of cave dimensions (and other variables related to their features).

The problem is that 3 of these variables are: Length, Area, and Volume. These 3 are highly correlated as they basically are: Area = Length * Length and Volume = Area * Length.

I know using PCA I can find the variables which influence the most the whole set's variation.

Is that ok to perform PCA and pick only the most 'influential' variable to my further analysis? Say, Area for example (I think Area is the best one, not sure, just a guess).

I know PCA does not choose variables. Instead, it creates new ones. However, I know the PC1 (being the major 'influencer' on the variation) is formed by loadings of the previous variables.

Does it make sense to pick then the variable which contributes with the most loadings on PC1?

• @amoeba I think that there are special features here (dimensional analysis, use of cave data) that make this sufficiently different. Jul 16, 2018 at 9:01
• Post the correlations between the three variables. Jul 16, 2018 at 9:24
• @NickCox Your answer is good (+1) and after seeing it I am certainly fine if this thread stays open. Jul 16, 2018 at 9:36

The question and some other answers seem to assume or imply that caves have simple shapes such that length, area and volume are broadly equivalent choices that just happen to have different dimensions. I don't have the data but on general geological (geomorphological) grounds I doubt that is the case.

Caves in practice are likely to be much more irregular in shape than say bricks or eggs or other volumes. But even if it were true as a first approximation that say

Area $$= b$$ Length$$^2$$

or

Volume $$= c$$ Length $$\times$$ Area

these are multiplicative relationships that won't play well with a correlation and PCA framework. You would almost certainly be much better off working with log Length, log Area, log Volume. Then your first approximations don't feature squares or cubes as powers. Log scales would also, I guess, work much better with what are likely to be highly skewed distributions with outliers present too. (Most caves are small, but a few are enormous.)

Note that you really don't need PCA to know what works best here. You can and perhaps should just look directly at correlations and scatter plots.

Posting (example) data would allow illustration. It seems that your main goal is using size variables as predictors, so telling us more about what you want to do would be a good idea.

There is a broader point here that statistical analysis should be informed by dimensional analysis when it applies. For details of a splendid article on this by David Finney (1917$$-$$2018) see https://www.jstor.org/stable/2346969

• As it happens, diagrams in the media from the recent rescue in Thailand should have underlined that caves are irregular and e.g. that the "height" can both vary a lot and be $\ll$ any horizontal dimension. Jul 16, 2018 at 14:45
• You are right. Caves do not have an shape that allow us using that simple equation. But I thought people would realize that was just an ''example'' of how things behave. Guess I was wrong. You are right again. Some caves are huge! Most of them are small sized. I may post some plots and data. Should I edit my question then? You are once again right. I want to use cave features, such as size ( 3 dimensions), water percolation, lithology, etc., to cluster my dataset and see if those caves may be divided in groups. Jul 16, 2018 at 14:46
• By all means edit your question to show example data, or even all of them if posting is practical. Copying and pasting say 200 observations with 8 variables is perfectly practical here. Jul 16, 2018 at 14:48
• I will accept your answer and my literature review. There is some discussion about ways/methods to measure cave dimensions. Some suggest the area as the best one due to the fact that it is less susceptible to measurement errors than the volume is. And, as mentioned, some caves are huge, some are small. Log(Area) kind of helps a lot solving this differences. Jul 19, 2018 at 14:26

PCA doesn't do variable selection. Instead, it does dimensionality reduction, which isn't the same. In your example, PCA might tell you that actually $3L + 5A - 2V$ explains a lot of the variation in your data, but no other combinations of $L, A, V$ do anything useful whatsoever.
This may of course solve the problem you have, but it doesn't tell you to that only one of the variables is useful.

As a side note, having variables which are correlated in this way isn't necessarily a bad thing and can in fact be a good thing.
If you want to do linear regression with your features, then you mostly need to be concerned about variables for which there is a linear relationship. But in your case there is a polynomial relationship between the variables, so you should be fine.
Further, polynomial regression is often very useful if the original relationship wasn't really linear. In your case, using $L, A, V$ as variables would roughly correspond to doing a polynomial regression with $L, L^2, L^3$.
To get an idea which of $L, A, V$ would be a good variable to include, you could look at a scatterplot of $L, Y$ and look whether it looks linear, quadratic or cubic.

I think the variables that in this case explicate the most variance could be Length, since ideally you should be able to represent all your features in relationship to this one.

Furthermore PCA just describes the data as a new set of variables,or axes, (that may not be your initial one) that maximize the variance in your dataset. So I think in this case you could just keep the variable "Length" since the variance of the others it is just a derivation of the variance on the variable "Length".