3
$\begingroup$

I have a data set with about 50 variables but only 10 observational units. This is due to the underlying science, it's difficult and expensive to increase the sample size. I think a large number of the variables are highly correlated, in terms of factor analysis I think there are only maybe 5-10 underlying factors. I want to group the variables based on their commonality (covariance or something similar). Is there a way to do this?

My first thoughts were factor analysis and hierarchical clustering. For factor analysis the sample size is obviously a huge problem. Similarly for hierarchical clustering I simulated data in R with small n and a large number of variables in such a way that I knew which ones were similar but with such small sample sizes the clusters are very unstable from one simulation to the next and are often spurious.

In short, is there a good way to group variables when your sample size is smaller than your number of variables?

$\endgroup$
2
$\begingroup$

One of the simplest approaches is hierarchical variable clustering using a similarity matrix. By default I take the similarity matrix to be made up of squared Spearman rank correlation coefficients. This is implemented in the R Hmisc package varclus function. I highly recommend sparse principal components as an alternative to this, which does clustering and scoring together. There are at least two R packages for this.

$\endgroup$
2
$\begingroup$

This is called low sample-size problem and it is solved through dimensionality reduction. Use them e.g. PCA, MDS, etc. And apply your analysis on them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.