# Clustering of multivariate data [closed]

Please I am about to cluster some data based which have about 15 different columns all of which are numbers(Some categorical while some are measurements) also some of my values are missing in some columns . Please can you give me pointer on how to go about it.

I have previously explored the clustering with weka but I am not sure about the way weka implements so I am going the R route.

What I know : I already know about Principal components analysis at least in theory. But is this necessary in all clustering of multiple columns . It will go a long way if anyone could provide me a link to a tutorial on this because Quick-R has for just 2 variables.

A sample of my dataset is listed below

1,64,9,30,33,2,3,1,6,1,5,-3.62,-3.71,-2.73,1
2,61,4,30,33,2,3,2,7,4,4,-3.62,-3.71,-2.00,1
3,49,4,18,21,2,3,2,8,17,18,-3.68,-3.88,-2.00,1
4,40,4,10,12,2,2,2,24,20,23,-3.32,-3.42,-2.00,1
5,43,9,10,12,2,2,1,2,1,29,-3.12,-3.19,-2.73,1
6,52,9,16,19,2,3,2,35,34,35,-3.33,-3.26,-2.95,1
7,46,4,15,18,2,3,2,8,40,42,-3.59,-3.50,-2.00,1
8,40,4,10,12,2,2,2,24,20,46,-2.45,-2.69,-2.00,1


First thing you'll want to do is deal with your missing data. Some stats packages will "deal with it" for you, but usually don't tell you when or how it's being done. A common approach is to replace missing values with the grand mean, or perhaps the mode for categorical data--or eliminate the data point altogether.

Your intuitions are right, PCA is not the only way to go, but it probably is the best approach here. If you happen to have a weighting scheme for your features (e.g., treating them all equally or according to a weighted model), you can compute a multivariate index for each item and do k-means clustering. The real advantage of this is that the computations are much simpler. If computation is not an issue, I'd opt for PCA instead, using a correlation matrix of your data.

Here's a tutorial I found using a quick google search, but full disclosure: I haven't done PCA in R.

R tutorial on PCA

• Some care is advised in doing PCA with the categorical variables! If they are purely categorical (and not ordered), perhaps they can be re-expressed as indicators.
– whuber
Aug 21 '11 at 20:57
• @whuber very good point. in my field (psychology) factor analysis is commonly done with likert scales-- so if you can arrange the categories as if they were ordinal, i think it should be ok. otherwise, i'm not sure what the solution is.
– Jeff
Aug 21 '11 at 21:05
• Thanks for the answer. Please what do you mean when you said I can arrange the categories as if they were ordinal. How can that be done ? Aug 21 '11 at 21:52
• @persistence i think what whuber is saying is that if you have multiple qualitative categories that are being denoted as numbers, your correlations will be spurious. an example would be assigning numbers 1,2,3 to categories 'blue' 'yellow' 'pink'. however, if your categories are things like 'small' 'medium' 'big', then they can be naturally represented ordinally using 1,2,3.
– Jeff
Aug 21 '11 at 22:01
• Thanks jeff. what happens to missing value would I replace with row means. Or what will be the best way to handle missing values ? Aug 22 '11 at 11:56

• For cluster analysis, mixed data types can be handled using Gower's universal similarity coefficient (DOI:10.2307/2528823). That works even with nominal data. Podani has published a modification of $$S_g$$ that uses the information in ordinal data better (DOI:10.2307/1224438). This coefficient also handles missing data (in effect by pairwise deletion, so that all available information is used). It has been shown that $$S_g$$ handles up to 25% missing data without too much bias (https://www.raco.cat/index.php/Questiio/article/viewFile/143135/194807). Imputation by regression from existing data has also been tried with success (https://www.researchgate.net/profile/Olivier_Gauthier2/publication/225554456_Missing_data_in_craniometrics_A_simulation_study/links/0c96052e276f03b78a000000.pdf)