I wonder whether it is possible to perform within R a clustering of data having mixed data variables. In other words I have a data set containing both numerical and categorical variables within and I'm finding the best way to cluster them. In SPSS I would use two - step cluster. I wonder whether in R can I find a similar techniques. I was told about poLCA package, but I'm not sure ...
$\begingroup$
$\endgroup$
6
asked Mar 12, 2012 at 20:55
-
1$\begingroup$ Isn't SPSS TwoStep designed for handling large datasets? (I provide a response to a related question here.) Otherwise, would my response to Can principal component analysis be applied to datasets containing a mix of continuous and categorical variables? be of any help? $\endgroup$– chlMar 13, 2012 at 20:26
-
$\begingroup$ Within the R package cluster there is ?daisy which will create a dissimilarity matrix for mixed data (Gower similarity coefficient). Then you can use ?agnes or other clustering functions. $\endgroup$– rhondaJan 23, 2013 at 17:00
-
1$\begingroup$ Don't confuse method with implementation. First look for a clustering algorithm that makes sense. Then look for an R package that implements it. $\endgroup$– shadowtalkerOct 2, 2014 at 1:33
-
$\begingroup$ Gower similarity can be used. $\endgroup$– ttnphnsFeb 22, 2016 at 7:13
-
$\begingroup$ @gung recently closed a very similar question I asked. I was told that my question was off topic because it was predominantly about software. This appears to be similarly about software. I'd be very interested to know why the rules here are being enforced inconsistently. Mind you, I think the question is informative, but the rules should be the rules. $\endgroup$– Weiwen NgFeb 27, 2019 at 0:55
Comments disabled on deleted / locked posts / reviews
|
Show 1 more comment
8 Answers
$\begingroup$
$\endgroup$
This may come in late but try klaR (http://cran.r-project.org/web/packages/klaR/index.html)
install.packages("klar")
It uses the non-hierarchical k-modes algorithm, which is based on simple matching as a distance function, so the distance δ between a variable m of two data points $x$ and $y$ is given by
$$ \delta(x_m,y_m) = \begin{cases} 1 & x_m \neq y_m,\\ 0 & \text{otherwise} \end{cases} $$
There is a flaw with the package, that is if two data points have the same distance to a cluster-center, the first in your data is chosen as opposed to a random point, but you can easily modify the bit in the code.
To accommodate for mixed-variable clustering, you will need to go into the code and modify the distance function to identify numeric and non-numeric modes and variables.
$\begingroup$
$\endgroup$
Another appealing way of handling variables of mixed types is to use the proximy/similarity matrix from Random Forests: http://cogns.northwestern.edu/cbmg/LiawAndWiener2002.pdf. This faciliates a unified way of equally treating all variables (nevertheless, be aware of the variable selection bias issue). On the other hand, there is really no gold universal way of defining distance for variables of mixed types. It all depends on the application contexts.
$\begingroup$
$\endgroup$
2
You might use multiple correspondence analysis to create continuous dimensions from the categorical variables and then use them with the numerical variables in a second step.
-
1$\begingroup$ How would you treat numerical variables in MCA? Using discretization? $\endgroup$– chlNov 29, 2012 at 18:43
-
$\begingroup$ There are extensions of MCA which can include continuous variables, see for example homogeneity analysis homals $\endgroup$ Oct 6, 2017 at 9:56
$\begingroup$
$\endgroup$
Well, you certainly can. By making the categorical variables artificially numeric. Or using a distance-matrix based clustering (fpc can probably do that). The question you should first try to answer is: does it actually make sense?
answered Mar 13, 2012 at 0:15
$\begingroup$
$\endgroup$
3
You could use the universal similarity coefficient of Gower (see Sneath & Sokal 1973, pp 135-136), which for two OTUs $j$ and $k$ is $$S_G = \frac{\sum_{i=1}^n{w_{i,j,k} s_{i,j,k}}}{\sum_{i=1}^n{w_{i,j,k}}}$$ for all characters $i$.
The weight $w_{i,j,k}$ is either 1 or 0, depending on whether the the comparison is valid or not (missing data, absence of binary character in both OTUs). More complicated weighing schemes have been published.
$s_{i,j,k}$ is calculated for
binary variables: 1 for concordance, 0 for discordance (equivalent to Jaccard's coefficient if $w_{i,j,k}$ is set to 0 for concordant absences)
multistate characters(nominal or ordinal): 1 for equality, 0 else (equivalent to the simple matching coefficient)
cardinal character: $s_{i,j,k} = 1 - \frac{|X_{i,j} - X_{i,k}|}{R_i}$ with $R_i$ the range of character $i$ (either in the population or in the sample).
The nice thing about $S_G$ is that it can not only handle all types of data, but is also robust towards missing data. It also results in positive semi-definite similarity matrices, i.e., OTUs are represented by points in Euklidian space (at least if not too many data are missing).
The distance between OTUs can be represented by $\sqrt{1-S_G}$
-
$\begingroup$ Can you define what is "character" (and "cardinal character") in your answer? Do by that you mean variable/attribute/feature? Besides, I might add that Gower can be computed for ordinal variables without treating them as nominal ("multistate"), see. $\endgroup$– ttnphnsAug 27, 2017 at 12:41
-
$\begingroup$ Caracter, variable, feature are all synonyms. Cardinal means either interval or rational scale. $\endgroup$ Aug 29, 2017 at 9:04
-
$\begingroup$ Thank you for claring it. I just asked because your terminology is apparently not very common in statistics or machine learning: "character" is unusual, and what you call "cardinal" variable type is typically known as "scale" variable aka "metrical" variable, it opposed to categorical. $\endgroup$– ttnphnsAug 29, 2017 at 9:14
$\begingroup$
$\endgroup$
If possible values of categorical variables are not too many, then you may think of creating binary variables out of those values. You can treat these binary variables as numeric variables and run your clustering. That's what I did for my project.
$\begingroup$
$\endgroup$
0
k-prototypes clustering might be better suited here. It combines k-modes and k-means and is able to cluster mixed numerical / categorical data. For R, use the Package 'clustMixType'.
https://cran.r-project.org/web/packages/clustMixType/clustMixType.pdf
$\begingroup$
$\endgroup$
VarSelLCM package offers
Variable Selection for Model-Based Clustering of Mixed-Type Data Set with Missing Values
On CRAN, and described more in paper.
Advantage over some of the previous methods is that it offers some help in choice of the number of clusters and handles missing data. Nice shiny app provided is also not be frowned upon.
