I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered score for expression of a particular attribute frequency for each case. In many cases though, any given attribute is likely to be 0 (i.e. very sparse for most).

The cases typically fall into 2-4 natural groups, and I'm trying to figure out how to find out which high-expression attributes are most representative for each group and/or which ones are driving the distinctions between group memberships.

I've been experimenting with using correlation clustering and the resulting groups do appear to match the natural groups assigned by other means, so now I'm just looking for a way to "unpack" those clusters in terms of their attribute expressions to find out which attributes/variables are the most influential ones in each of my samples.

Given the number of variables involved, it seems like the usual approaches like PCA or discriminate or factor analysis would be very cumbersome. So far, I haven't really found much information on how to deal with variable influence that would fit situations when there are scores or hundreds of variables.

Any suggestions?

  • $\begingroup$ Can you be more precise with program/function used for correlation clustering ? $\endgroup$ May 21, 2013 at 9:33
  • $\begingroup$ @lmorin - I'm using R, and trying out various clustering methods using variations on Dist=as.dist(1-cor). $\endgroup$
    – JSCard
    May 21, 2013 at 14:01
  • $\begingroup$ I've started looking into the package randomForest, which has some interesting tools for evaluating variable importance. I'd like to understand the more standard metrics on variable influence and importance in high-dimension correlation, though. $\endgroup$
    – JSCard
    May 22, 2013 at 14:16


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.