After performing an Hierarchical Clustering on Multiple Correspondences Analysis, I want to test the differences between my variables amongst the clusters. The goal is to see which variables is more related to one cluster. Which test can I do in order to have those results ? Just a quick reminder, in my dataframe, I just have categorical variables.

Best regards. BNDKE

  • $\begingroup$ Can you try khi 2 test? $\endgroup$ Commented Aug 3, 2021 at 13:32
  • 2
    $\begingroup$ @Abdoul That would not be legitimate, because the clusters were obtained with the same data used for the testing. The p-values will be incorrect. $\endgroup$
    – whuber
    Commented Aug 3, 2021 at 13:38
  • 1
    $\begingroup$ Can you please explain what you mean by "test the differences between my variables"? The word "test" seems to trigger a hypothesis testing association by many readers here on CV, but it seems to me that you are not interested in p values, but are looking for something different and that you use the word "test" in its colloquial meaning, not its statistical meaning. $\endgroup$
    – cdalitz
    Commented Aug 3, 2021 at 15:25
  • $\begingroup$ @cdalitz you made a good point. In fact, I want to know if there are any metrics than I can implement to determine which variables is more related to one cluster and not another. $\endgroup$ Commented Aug 4, 2021 at 8:26

1 Answer 1


This is a feature selection problem in classification with categorial features: your cluster labels are the classes, and the features are your catgorial variables.

There are two common approaches to this problem:

  1. Test each variable for independence from the cluster label, e.g. with a $\chi^2$ test of independence. Beware however the comment by @whuber that the p-values must not be interpreted as error probabilities for $H_0$. Nevertheless, you can use the p-values $p_i$ for each variable to compute score values, e.g., $1-p_i$ or $1/(1+p_i)$.
  2. Maximize the joint mutual information between the selected features and the target variable (cluster label), e.g., as described by Bennasar, Hicks & Setchi (2015).

Method 1. is only a scalar method, i.e., it does not consider interactions, but it has the advantage of being easy to implement. As method 2. also takes interactions between variable sinto account, it is, theoretically, preferable.

Both methods are filter methods which do not rely on a classification algorithm and do not optimize classification error. If you care about classification accuracy (which your question seems to indicate), a possibly preferable approach would be to use a wrapper method around a classification algorithm that can handle categorial features, e.g., random forest. If you have a large number of features and training samples, Boruta is a very popular method:

M.B. Kursa, W.R. Rudnicki: "Feature Selection with the Boruta Package." Journal of Statistical Software 36,11, pp. 1-13 (2010)

  • 2
    $\begingroup$ It's probably worthwhile saying what @whuber had already mentioned in the comment for the question itself, namely that the p-value from an independence test might be used as some kind of "score", but it wouldn't be valid to interpret it as a p-value of a test here; this is made invalid by the fact that the clusters were formed based on the same data. $\endgroup$ Commented Aug 4, 2021 at 16:02
  • $\begingroup$ @lewian Thanks for the hint, which I have added in my answer. An even more serious problem with the $\chi^2$ test is that it evaluates each variable in isolation. I have additionally mantioned this shortcoming. $\endgroup$
    – cdalitz
    Commented Aug 4, 2021 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.