0
$\begingroup$

I need an approach to assign weights to variables based on the cluster. I am not trying to learn the clusters, as I already have them, but I would just like to now understand how "important" each variable is to the cluster. Is there any way of determining weights per variable per cluster?

I thought I could use a multi-class classifier and learn the variable importance and use them as weights, but these weights would then be global across all clusters, and not cluster specific.

In another attempt, I could probably do something like a logistic regression one-versus all, and use the coefficients there as weights. But I'm wondering if this even makes sense.

Any idea on how to learn how important features are to clusters?

$\endgroup$
1
$\begingroup$

You say that you already have the clusters. If you have enough data points, I would split them and perform a principal component analysis.

This will give you a unique component for each cluster. This vector will not be same as the original but each cluster would get a different configuration of vectors.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ If I had 10 clusters, with 10 sets of principal components, how would I validate that this approach is reasonable? I'd imagine I could do 10-fold-cross-validation, and on the testing set, compare the assigned label with the true labels using something like the adjusted Rand Index? $\endgroup$ – Jane Wayne Jan 23 '18 at 6:55
  • $\begingroup$ But still, that doesn't give me weights for the variables. $\endgroup$ – Jane Wayne Jan 23 '18 at 6:56
  • 1
    $\begingroup$ You can use the eigenvalues as "weights" of the component vectors $\endgroup$ – knk Jan 23 '18 at 13:53
  • $\begingroup$ I am not sure I understand your problem, do you want to learn the importance of features or perform a classification? If you want to perform classification, then train a classifier on the new feature vectors and compare its performance to the old feature vectors. $\endgroup$ – knk Jan 23 '18 at 13:58
1
$\begingroup$

I am inclined to think your initial intuition about how to do this was on most reasonable. Use a multi-class classifier and determine which variables separate the clusters best.

Going cluster-by-cluster and trying to decide which variables appear to be important is going to yield little value for understanding which variables determine membership in any given cluster. This approach is akin to trying to predict a constant. You will need to see how the variables distinguish between the clusters to understand what is important.

One approach to doing this might be a multinomial logit-based dominance analysis, as you suggest, using the cluster memberships as dependent variable. If classification is really good though, that could cause convergence problems for the maximum likelihood estimator (assuming that's what you would use).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.