I have performed consensus clustering on a data set (N=2500), which resulted in a best value of k=3. The data set is from the medical domain, and the features represent demographic (e.g. age, sex) and medical data (presence of comorbidities, medicines etc). I intend to define clusters within this data set that represent sub-phenotypes. These clusters should have some characteristics that define them, e.g. cluster 1 is defined by a high age and a high prevalence of Diabetes. The data consists of both numerical and categorical data.

Now, I have the clusters but I'm not sure how I would statistically validate these clusters. Should I just take all the features and check whether or not they significantly differ from the patients outside the cluster? E.g. for continuous/numeric variables like age, use the Mann-whitney test to test whether or not the ages from patients in cluster a significantly differ from all other clusters? And then test the categorical features with a Person's Chi-square test? Or is there a better way of doing this? Intuitively this makes sense to me, but there are probably caveats I'm not aware of.

I read about this local article about this topic. However, I think the answers to this post do not yet answer my questions. Bootstrapping sounds like a way to make things more robust, but I suppose I already am bootstrapping since I'm using consensus clustering. I also found this relevant, local article, but it has no answers.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.