I am trying to cluster some big data by using the k-prototypes method. I am unable to use K-Means as I have both categorical and numeric data. I have been using the package "clustMixType" and have been able to create clusters if I define what k value I want. I want to find the optimal k value though and can't find anything on this online already.
2 Answers
As far as I know there's no generic optimal k.
It depends a lot on your dataset and your goal. A lower K would yield more fuzzy prototypes but would generalize better. There are always trade-offs
One way to pick K is to plot the data, and look at it. Even then you might want to try other values to see if they work better for your application.
You may use the code as below to plot the elbow curve. The input to the code below is the .
data <- <input the data here>
# Elbow Method for finding the optimal number of clusters
set.seed(123)
# Compute and plot wss for k = 2 to k = 15.
k.max <- 15
data <- na.omit(data) # to remove the rows with NA's
wss <- sapply(1:k.max,
function(k){kproto(data, k)$tot.withinss})
wss
plot(1:k.max, wss,
type="b", pch = 19, frame = FALSE,
xlab="Number of clusters K",
ylab="Total within-clusters sum of squares")
clue
package. $\endgroup$clustering criterions
akaclustering validation indices
. $\endgroup$