2
$\begingroup$

I'm thinking to run PCA within a pre-processing step before an other PCA which will actually be used to analyse sub-sampled data. I thought it might be useful to sub-sample the original set of variables by choosing the ones which contributed the most to the best components of the pre-processing PCA and then, do the second run of a PCA this time to analyse (mostly vizualize) the subset of data with the most contributing variables (a decent number of them). In my view, the variables which contribute most to the best PCs, most contribute the variance of the high dimensional data vector. So they are able to capture more information from the raw data. So they are more informative.

Does that make any sense? Thanks

$\endgroup$

1 Answer 1

4
$\begingroup$

Yes, to some degree this makes sense. You don't want to just throw everything into a PCA and see what comes out, there is some degree of evaluation related to what data goes into a PCA. For a PCA there are some generalized assumptions that are good to follow to make sure the analysis is as robust as possible.

First and foremost you need to make sure PCA is the analysis that you want as in understanding what a PCA does to the data provided and is that your intended end result. For instance, if you are trying to maximize explaining the variance in your dataset then PCA is what you want. But if you are trying to figure out have things are related to one another (scaled or otherwise) then something like an NMDS would be better.

Generalized Assumptions

  • Multiple Variables: This one is obvious. Ideally, given the nature of the analysis, multiple variables are required to perform the analysis. Moreover, variables should be measured at the continuous level, although ordinal variable are frequently used.

  • Sample adequacy: Much like most (if not all) statistical analyses to produce a reliable result large enough sample sizes are required. Generally a minimum of 150 cases (i.e. rows), or 5 to 10 cases per variable is recommended for PCA analysis. Some have suggested to perform a sampling adequacy analysis such as Kaiser-Meyer-Olkin Measure (KMO) Measure of Sampling Adequacy. However, KMO is less a function of sample size adequacy as its a measure of the suitability of the data for factor analysis, which leads to the next point.

  • Linearity relationships: It is assumed that the relationships between variables are linearly related. The basis of this assumption is rooted in the fact that PCA is based on Pearson correlation coefficients and therefore the assumptions of Pearson’s correlation also hold true. Generally, this assumption is somewhat relaxed…even though it shouldn’t be…with the use of ordinal data for variable.

  • Co-linearity (not in the original blog post): Things to avoid to some degree are parameters with high co-linearity. In my field of biogeochemistry/water quality I don't like including variables that are used to calculate another variable and the calculated variable. As an example, total nitrogen (TN) can be calculated from the sum of nitrate-nitrite (NOx) and total kjeldahl nitrogen (TKN; i.e. TN = NOx + TKN). Therefore, I would include TN or NOx and TKN but not all three. Including all three could affect the variance explained and lead to autocorrelation down the statistical road.

  • No significant outliers: Like most statistical analyses, outliers can skew any analysis/ In PCA, outliers can have a disproportionate influence on the resulting component computation. Since principal components are estimated by essentially re-scaling the data retaining the variance outlier could skew the estimate of each component within a PCA. Another way to visualize how PCA is performed is that it uses rotation of the original axes to derive a new axes, which maximizes the variance in the data set. In 2D this looks like this:

Here is a link to an old blog post I put together discussing PCA, and a little how-to using R. Hopefully, this is helpful.

$\endgroup$
1
  • 1
    $\begingroup$ Thanks Paul! My Prof. rejected the preprocessing PCA, but I don't have expertise in the dataset. I only excluded variables ~derived from others but he wanted I sub-sample way more so the (main) PCA results can be interpreted. Hope to see more answers here too. $\endgroup$
    – Zuko36
    Commented Sep 7 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.