I have the time series of 16 water quality parameters, and after standardizing them using the zscore method, I performed principal component analysis. These are my eigenvalues [7.62675203795075 2.25806327090482 1.72428547604366 1.44517173004117 1.30223669986535 1.18683140226834 1.01121663757535 0.840810909788259 0.683733271693109 0.641289626275986 0.427483271093487 0.367733123021385 0.243180116969796 0.216284282138318 0.132868635217870 0.0705244464795644]. As you can see, the Eigenvalue of the first PC is very different from the rest of the PCs, which I have not seen in previous studies, and it decreases with smaller intervals. What factors have caused the results to be like this? Is my data not suitable for principal component analysis? I also did the KMO test and the value was 0.91.
-
$\begingroup$ The KMO test assesses whether the data allow for the reliable extraction of factors; with components the KMO is not needed. I.e., PCA != EFA. $\endgroup$– rolando2Commented Jul 19 at 12:58
2 Answers
There are a few factors that can cause the first PC to be very dominant. Some simple ones are:
- We have a highly correlated sample, i.e. there is a lot of redundant information in the data and therefore the first PC reflects that.
- We really have a very dominant underlying trend (e.g. temperature seasonality) so indeed the first PC "should dominate" in terms of variance.
- We mess up the normalisation. I see that the OP explicitly says "standardizing them using the zscore method". More generally though, if we have features in different scales, or by accident when normalising we miss normalising a feature, we can be led to a situation that a single feature has such larger scale than everything else that dominates the sample variance. Subsequently, the first PC will be dominant too.
The above being said, for the sample shown in the OP question in particular, I would say that the first PC is not that dominant. It accounts for roughly 37% of the variance, which is not that much considering we have (only) 16 dimensions anyway.
Finally, on the "is my data (not) suitable for PCA" side-question: All data is suitable for PCA; it is a question what we do with the PCA results and whether they are suitable/insightful for our respective research question.
-
$\begingroup$ I was nodding agreement all through -- until the very last sentence. On the contrary, the eigenvalues and eigenvectors carry a lot of information! $\endgroup$– Nick CoxCommented Jul 19 at 14:19
-
$\begingroup$ @NickCox: Thank you for your comment. Hmm... maybe I am misexpressed myself but let's say here, where we get $\lambda_{1-3}$ being 7.62, 2.25 and 1.72. What do they mean on their own? OK, that the first PC has ~3.5x higher variance than the second PC, is that good, bad, indifferent? Without context, the values "on their own mean very little". Does an eigenvector show correlation or anti-correlation between features $x_a$ and $x_b$? Is that known already, so it is a trivial result? Is it an exceptional insight? "On their own mean very little". $\endgroup$ Commented Jul 19 at 16:43
-
$\begingroup$ Would you find it better if the last sentence read "(...) and eigenvectors values on their own (...)"? (Thanks again for keeping my game from going sloppy.) $\endgroup$ Commented Jul 19 at 16:45
-
$\begingroup$ Well, I would find it better if you deleted that last sentence or replaced it at greater length.... $\endgroup$– Nick CoxCommented Jul 19 at 19:03
This can mean that one linear combination of your 16 predictors contains most of the information in the data that is about your response variable.
(But without knowing more about the dataset I can't say more than that.)