Is the matrix dimension important for performing a valid PCA?

Question

If $X$ is a $m × n$ matrix, where $m$ is the number of measurement types (variables) and $n$ is the number of samples, would it be correct to perform a PCA on a matrix that has $m \geq n$ ? If not, please provide some arguments why would this be a problem.

I remember having heard that doing such an analysis would be invalid, but the Wikipedia page for PCA doesn't mention a low $n/m$ ratio as being a potential limitation for using the method.

Please note that I am a biologist and aim at a more practical answer (if possible).

Answer 1

Matrix dimension by itself have little todo with PCA validity. What will change is the interpretation of your data and it's all dependent on how you want to use the result.

PCA is very powerful to use to find anomalys or outliers in your data. Maybe you have performed an experiment on two different days, used different machines in the experiment etc. If the purpose is to get an overview of data PCA is one of the most efficient way to do that regardless of any n/m ratios.

If your main interest is to investigate clusters or relations between samples, then #variables are not very important. (But other type of statistics on the result might be important if #samples are low).

If you look at individual variables, then they will be less reliable if you have few samples. However, this is a problem you will have with any other method as well. If you find patterns among variables that make sense then you should certainly not disregard your findings because you have a low n/m ratio. However, few observations are almost allways problematic and should lead to caution in interpretation and the more samples you have the less important is the #sample/#variable relation.

Answer 2

I do not think that you would get any useful information from such an analysis, as lore in my subject area (psychology) suggests a 10;1 ratio in favour of n as a precondition. In some circumstances (where communalities are high) you can get away with 5 or 3 to 1, but a ratio of less than 1 is probably a recipe for disaster.

Answer 3

PCA of variables. Number of observations n is low relative to number of variables. 1) Mathematical aspect. Whenever n<=m correlation matrix is singular which means some of last m principle components are zero-variance, that is, they are not existant. This is not a problem to PCA, generally speaking, since you could just ignore those. However, many software (mostly those uniting PCA and Factor Analysis in one command or procedure) will not allow you to have singular correlation matrix. 2) Statistical aspect. To have your results reliable you must have correlations reliable; that requires considerable sample size which always should be larger than number of variables. They say, if you have m=20 you ought to have n=100 or so. But if you have m=100 you should have n=300 or so. As m grows, minimal recommended n/m proportion diminishes.