Let's say you have a density plot of data in 2D (or even 1D). Surely there are algorithms which infer the number of clusters which exist in the data without users having to explicitly set this number (e.g. like with traditional k-means).
How does one use unsupervised learning to solve this?
Whichever way you go, you will still need to justify on what basis you (or an algorithm) set the number of clusters.
There is an R package called optClust assesses a wide range of clustering algorithms and a set range of number of clusters.
Here's a usage example from the documentation:
library(optClust)
## Obtain Dataset
data(arabid)
## Analysis of Count Data using Internal and Stability Validation Measures
count1 <- optCluster(arabid, 2:4, clMethods = "all", countData = TRUE)
summary(count1)
This utilises the clValid and RankAggreg packages (among others). The clValid package has a several validation measures which each clustering algorithm and number of clusters is assessed against. The RankAggreg package then is used to "combine these multiple rank lists together in some way to form a single list that best represents the original rankings" (Sekula, 2010, p 29) - giving a ranked list of algorithm and number of clusters.
There is also the NbClust package that "provide[s] an exhaustive list of validity indices [(30 in total)] to estimate the number of clusters in a data set". Suitable for k-means or HAC.
Example from documentation:
data <- iris[, -5]
diss_matrix <- dist(data, method = "euclidean", diag = FALSE)
NbClust(data, diss = diss_matrix, distance = "NULL", min.nc = 2, max.nc = 10, method = "complete", index = "alllong")
Coming back to my original point, both of these options have default parameters that may not suit your research question / dataset.
You can have a look at the answers to this question for explanations of different methods for determining the number of clusters. There are also many other posts on the topic.
