2
$\begingroup$

I have observations with 6 individual-level characteristics from 2000 units in 56 groups (villages). I know that within groups, characteristics are strongly related as individuals influence each other.

Does it still make sense to apply PCA to the 6 individual characteristics in order to single out the 2 most important components?

Improve this question
$\endgroup$
7
  • $\begingroup$ @whuber has a nice long answer on whether it makes sense to apply PCA on correlated variables, and whether to remove them. See here: stats.stackexchange.com/questions/50537/… $\endgroup$
    – Maximilian Aigner
    Commented Dec 20, 2016 at 11:43
  • 1
    $\begingroup$ Thank you, but it's not about the variables being correlated, but about the observation not being sampled i.i.d from a population. $\endgroup$
    – sheß
    Commented Dec 20, 2016 at 11:45
  • 2
    $\begingroup$ If you have many groups with many observations, I'd argue you can apply PCA without regret, though perhaps you could try standardizing within each grouping instead of standardization with the whole dataset. Something like random effects PCA sounds interesting as well, there must be something along those lines already. $\endgroup$
    – Firebug
    Commented Dec 20, 2016 at 11:54
  • 1
    $\begingroup$ In many fields e.g. meteorology, climatology doing PCA on data in space and/or time is the key, or at the very least, a key application. It's as well to look around to see that dependence is in absolutely no sense a barrier to PCA being useful. I don't know a book I really like on PCA but Jolliffe's book does have many such examples. springer.com/gb/book/9780387954424 $\endgroup$
    – Nick Cox
    Commented Dec 20, 2016 at 12:39
  • 3
    $\begingroup$ The crux is whether inference is intended as well as description or explanation or just transformation. If it is, then you need to be mindful of assumptions about generating processes. An analogue is that logarithmic or other transformation of single variables can be useful regardless of whether data are dependent. As PCA is, arguably, a multivariate transformation only (scope for dissent there), I would argue similarly. $\endgroup$
    – Nick Cox
    Commented Dec 20, 2016 at 12:42

1 Answer 1

Reset to default
2
$\begingroup$

A few thoughts: First, since you have only 6 variables, are you sure you need PCA? If you expect to get 2 components, you haven't gained that much simplicity.

Second, if you decide that you do want to do it, the wisdom of doing it per group vs. all at once depends on both the question you want to ask and the answer you think you will get. That is, if you suspect that the structure will be different in each group, you should do it by group to see if you are right. You might even wind up clustering the groups by their PCA structure (that would be an interesting problem). If the structures are all different, then mixing the groups may result in mush. You risk the "Switzerland problem" - i.e. On average, Switzerland is flat. Throw the mountains into the lakes and it balances out.

Improve this answer
$\endgroup$
6
  • 5
    $\begingroup$ As a geographer and geomorphologist some of the time, I take the Switzerland analogy as provocative and evocative, but have to point out that it's not literally true. Lakes and mountains are not equally numerous and not equal in volumetric characteristics! $\endgroup$
    – Nick Cox
    Commented Dec 20, 2016 at 12:45
  • 1
    $\begingroup$ @NickCox I can't see how that analogy is wrong. On average, any place is flat. $\endgroup$
    – Firebug
    Commented Sep 28, 2018 at 11:19
  • 3
    $\begingroup$ @Firebug I was just having fun, pedantically or otherwise. By lakes, I do mean lakes, depressions containing water, not places with altitude less than the mean. Your new statement is wrong too as for slopes in a geographer's sense don't average, in a landscape, to zero. $\endgroup$
    – Nick Cox
    Commented Sep 28, 2018 at 11:28
  • 1
    $\begingroup$ I would agree that Switzerland is locally flat. Any place I were it seemed flat, four legged table would stand not rocking etc. At the same time you could see mountains from any point $\endgroup$
    – Aksakal
    Commented Apr 4, 2019 at 21:19
  • 1
    $\begingroup$ Gradient in geography and Earth sciences is usually a scalar field, the tangent of the surface slope at any point in the steepest direction. (It's more common for many purposes to use angle in degrees, but that's not material to the point.) $\endgroup$
    – Nick Cox
    Commented Aug 12, 2019 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.