1
$\begingroup$

Please I am about to cluster some data based which have about 15 different columns all of which are numbers(Some categorical while some are measurements) also some of my values are missing in some columns . Please can you give me pointer on how to go about it.

I have previously explored the clustering with weka but I am not sure about the way weka implements so I am going the R route.

What I know : I already know about Principal components analysis at least in theory. But is this necessary in all clustering of multiple columns . It will go a long way if anyone could provide me a link to a tutorial on this because Quick-R has for just 2 variables.

A sample of my dataset is listed below

1,64,9,30,33,2,3,1,6,1,5,-3.62,-3.71,-2.73,1
2,61,4,30,33,2,3,2,7,4,4,-3.62,-3.71,-2.00,1
3,49,4,18,21,2,3,2,8,17,18,-3.68,-3.88,-2.00,1
4,40,4,10,12,2,2,2,24,20,23,-3.32,-3.42,-2.00,1
5,43,9,10,12,2,2,1,2,1,29,-3.12,-3.19,-2.73,1
6,52,9,16,19,2,3,2,35,34,35,-3.33,-3.26,-2.95,1
7,46,4,15,18,2,3,2,8,40,42,-3.59,-3.50,-2.00,1
8,40,4,10,12,2,2,2,24,20,46,-2.45,-2.69,-2.00,1
Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

First thing you'll want to do is deal with your missing data. Some stats packages will "deal with it" for you, but usually don't tell you when or how it's being done. A common approach is to replace missing values with the grand mean, or perhaps the mode for categorical data--or eliminate the data point altogether.

Your intuitions are right, PCA is not the only way to go, but it probably is the best approach here. If you happen to have a weighting scheme for your features (e.g., treating them all equally or according to a weighted model), you can compute a multivariate index for each item and do k-means clustering. The real advantage of this is that the computations are much simpler. If computation is not an issue, I'd opt for PCA instead, using a correlation matrix of your data.

Here's a tutorial I found using a quick google search, but full disclosure: I haven't done PCA in R.

R tutorial on PCA

Improve this answer
$\endgroup$
6
  • 3
    $\begingroup$ Some care is advised in doing PCA with the categorical variables! If they are purely categorical (and not ordered), perhaps they can be re-expressed as indicators. $\endgroup$
    – whuber ♦
    Commented Aug 21, 2011 at 20:57
  • $\begingroup$ @whuber very good point. in my field (psychology) factor analysis is commonly done with likert scales-- so if you can arrange the categories as if they were ordinal, i think it should be ok. otherwise, i'm not sure what the solution is. $\endgroup$
    – Jeff
    Commented Aug 21, 2011 at 21:05
  • $\begingroup$ Thanks for the answer. Please what do you mean when you said I can arrange the categories as if they were ordinal. How can that be done ? $\endgroup$
    – persistence911
    Commented Aug 21, 2011 at 21:52
  • 1
    $\begingroup$ @persistence i think what whuber is saying is that if you have multiple qualitative categories that are being denoted as numbers, your correlations will be spurious. an example would be assigning numbers 1,2,3 to categories 'blue' 'yellow' 'pink'. however, if your categories are things like 'small' 'medium' 'big', then they can be naturally represented ordinally using 1,2,3. $\endgroup$
    – Jeff
    Commented Aug 21, 2011 at 22:01
  • $\begingroup$ Thanks jeff. what happens to missing value would I replace with row means. Or what will be the best way to handle missing values ? $\endgroup$
    – persistence911
    Commented Aug 22, 2011 at 11:56
0
$\begingroup$

That depends on your data:

  • Do the missing data occur randomly, or is "missing" related to some other variable? For example, in one of my problems I deal with student success. If a student is dismissed after the first semester, then data for the following semester are missing. However, it is very unlikely that the average grades of these students, had they been allowed to continue, in the following semesters would have been the same as the average grade of those students that passed first semester. Thus, the data are missing non-randomly and need extra careful handling in order to avoid bias.
  • How much co-linearity is in your data, in other words, what is the rank of the data matrix? PCA will tell you that, and can also be used to reduce dimensionality if needed. For example, in a recent problem I had 170 variables, but only 24 significant eigenvalues. In such cases, it makes sense to do further analysis with the scores on these 24 components.
  • PCA is in my experience totally unsuitable for nominal data, but works with ordinal and binary, and of course shines with cardinal.
  • For cluster analysis, mixed data types can be handled using Gower's universal similarity coefficient (DOI:10.2307/2528823). That works even with nominal data. Podani has published a modification of $S_g$ that uses the information in ordinal data better (DOI:10.2307/1224438). This coefficient also handles missing data (in effect by pairwise deletion, so that all available information is used). It has been shown that $S_g$ handles up to 25% missing data without too much bias (https://www.raco.cat/index.php/Questiio/article/viewFile/143135/194807). Imputation by regression from existing data has also been tried with success (https://www.researchgate.net/profile/Olivier_Gauthier2/publication/225554456_Missing_data_in_craniometrics_A_simulation_study/links/0c96052e276f03b78a000000.pdf)
Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.