# Clustering data based on correlation

I have a dataset where each row represents a sample and each sample is described by its chemical composition. You can see the 10 first rows of the dataset in figure 1.

Figure 1 - Each row represent a sample and each sample is decomposed into the 17 different chemical compounds and the total (all values are given in percentage)

First I found the correlation between the samples and made the correlation matrix shown in figure 2.

But what I really want to cluster the chemical compounds that are more likely to be found together in a sample.

• Absolutely, if you have a sensible idea of the size of $k$. Turn your correlation matrix into a correlation based distance matrix (see link given in the answer) and run the $k$-means algorithm. Also read my link about standardisation. – Bernhard Apr 3 at 14:05
• Ok, I misunderstood that, but the structure of the problem does not change. You will need to compute a correlation based distance matrix of the chemical components, then, before you do either hierarchical or $k$-means clustering. – Bernhard Apr 3 at 14:13
• Point 7 in the "Notes" paragraph looks promising. Now look for a clustering function that can take the output of pdist(X, 'correlation') as it's input. I am no Python user so I cannot give you more hints as to which combination of functions/methods will do the trick but I'd expect some hints to worthwhile functions (including the heatmaps with dendrograms in fig 3) in mycarta.wordpress.com/2019/04/10/… – Bernhard Apr 3 at 14:40