I have observations with 6 individual-level characteristics from 2000 units in 56 groups (villages). I know that within groups, characteristics are strongly related as individuals influence each other.

Does it still make sense to apply PCA to the 6 individual characteristics in order to single out the 2 most important components?

  • $\begingroup$ @whuber has a nice long answer on whether it makes sense to apply PCA on correlated variables, and whether to remove them. See here: stats.stackexchange.com/questions/50537/… $\endgroup$ – Maximilian Aigner Dec 20 '16 at 11:43
  • 1
    $\begingroup$ Thank you, but it's not about the variables being correlated, but about the observation not being sampled i.i.d from a population. $\endgroup$ – sheß Dec 20 '16 at 11:45
  • 2
    $\begingroup$ If you have many groups with many observations, I'd argue you can apply PCA without regret, though perhaps you could try standardizing within each grouping instead of standardization with the whole dataset. Something like random effects PCA sounds interesting as well, there must be something along those lines already. $\endgroup$ – Firebug Dec 20 '16 at 11:54
  • 1
    $\begingroup$ In many fields e.g. meteorology, climatology doing PCA on data in space and/or time is the key, or at the very least, a key application. It's as well to look around to see that dependence is in absolutely no sense a barrier to PCA being useful. I don't know a book I really like on PCA but Jolliffe's book does have many such examples. springer.com/gb/book/9780387954424 $\endgroup$ – Nick Cox Dec 20 '16 at 12:39
  • 2
    $\begingroup$ The crux is whether inference is intended as well as description or explanation or just transformation. If it is, then you need to be mindful of assumptions about generating processes. An analogue is that logarithmic or other transformation of single variables can be useful regardless of whether data are dependent. As PCA is, arguably, a multivariate transformation only (scope for dissent there), I would argue similarly. $\endgroup$ – Nick Cox Dec 20 '16 at 12:42

A few thoughts: First, since you have only 6 variables, are you sure you need PCA? If you expect to get 2 components, you haven't gained that much simplicity.

Second, if you decide that you do want to do it, the wisdom of doing it per group vs. all at once depends on both the question you want to ask and the answer you think you will get. That is, if you suspect that the structure will be different in each group, you should do it by group to see if you are right. You might even wind up clustering the groups by their PCA structure (that would be an interesting problem). If the structures are all different, then mixing the groups may result in mush. You risk the "Switzerland problem" - i.e. On average, Switzerland is flat. Throw the mountains into the lakes and it balances out.

  • 5
    $\begingroup$ As a geographer and geomorphologist some of the time, I take the Switzerland analogy as provocative and evocative, but have to point out that it's not literally true. Lakes and mountains are not equally numerous and not equal in volumetric characteristics! $\endgroup$ – Nick Cox Dec 20 '16 at 12:45
  • 1
    $\begingroup$ @NickCox I can't see how that analogy is wrong. On average, any place is flat. $\endgroup$ – Firebug Sep 28 '18 at 11:19
  • 3
    $\begingroup$ @Firebug I was just having fun, pedantically or otherwise. By lakes, I do mean lakes, depressions containing water, not places with altitude less than the mean. Your new statement is wrong too as for slopes in a geographer's sense don't average, in a landscape, to zero. $\endgroup$ – Nick Cox Sep 28 '18 at 11:28
  • $\begingroup$ I would agree that Switzerland is locally flat. Any place I were it seemed flat, four legged table would stand not rocking etc. At the same time you could see mountains from any point $\endgroup$ – Aksakal Apr 4 at 21:19
  • $\begingroup$ @Nick Cox: What is, mathematically, slopes in a geographer's sense? gradients? directional derivatives? ... $\endgroup$ – kjetil b halvorsen Aug 11 at 8:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.