1
$\begingroup$

I have a dataset from a questionnaire with over 10000 rows and 30 variables. I am trying to have an insight of the data so I tried to cluster similar items. I first made a dimension reduction technique (t-SNE) and then apply a clustering algorithm, Agglomerative Hierarchical Clustering. Then I got this visualisation:

enter image description here

So now I have clusters but I don't have any clue what these clusters show. My dataset is way too big to have a good view of what each cluster represents. So is there any technique/method to aim that without looking into each variable of each rows? Thank you

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

What you can do is to plot for each cluster the mean and SD / CI of all the variables. This will enable you to spot the big differences. For ex., suppose we have 3 cluster with some variables such as the following:

enter image description here

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you for your reply. Most of my variables are actually categorical variables. Do you think it is more wise to use the median or I should use the mean? $\endgroup$
    – Gaelle Sou
    Jan 16, 2019 at 14:24
  • $\begingroup$ The best way would be to standardize all the values is some way so that they are all on the same scale. After that it doesn't matter if you take the mean or median. $\endgroup$
    – user2974951
    Jan 16, 2019 at 14:34
  • $\begingroup$ And also, just to be sure that I interprete well the graph, in your graph, it shows that the 3 clusters mainly differ in the values of features 'disp' and 'hp'. Is that correct? Does the feature 'mpg' also counts or the difference is too small to consider it? $\endgroup$
    – Gaelle Sou
    Jan 16, 2019 at 15:17
  • $\begingroup$ @GaelleSou That is correct, and yes, the differences in mpg are too small since the CI's are overlapping to some degree. A close up (or using standardized data) would show more clearly how big of a difference there is. $\endgroup$
    – user2974951
    Jan 16, 2019 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.