What are suitable clustering algorithms for high dimensional data, where I do not have to input a predetermined number of clusters?
$\begingroup$
$\endgroup$
2
1 Answer
$\begingroup$
$\endgroup$
1
The most common clustering technique that meets your requirements would be DBSCAN. This finds points that are continuous by virtue of having shared nearest neighbors. There can be any number of clusters, and they can be of any shape. There are only two parameters to choose / enter: epsilon, a 'reachability' distance, and minpoints, the minimum number of points for the resulting set to be considered a cluster.
The problem (potentially) in high-dimensional space is the curse of dimensionality. That is, all points can become equidistant from all other points. Since DBSCAN uses distance directly, this could become a problem and require you to explore distances other than Euclidean or other remedies. Note that high dimensionality does not necessitate this problem, however. For more, see this excellent CV answer: Euclidean distance is usually not good for sparse data?
answered Jun 24, 2015 at 20:34
-
3$\begingroup$ To anyone coming back to this, you want to use HDBSCAN! github.com/scikit-learn-contrib/hdbscan $\endgroup$– N. McA.Commented Dec 26, 2017 at 23:45
two-step clusterandtwostep clusteron this site. This method can suggest the number of clusters at a halfway of the process. $\endgroup$