It is said that kMeans clustering works as long as we don't have clusters of differing
- and non-spherical shapes
I understand how one might check the sizes and densities of data points, but I am unsure how one would check for non-spherical clusters, especially if one merely has a conceptual understanding of a dataset.
So, Q1. Since most people with real world data sets would not know if they have non-spherical clusters, would it not be a mistake to apply kMeans?
Q2. What would be a more suitable choice of a clustering algorithm if we only want to partition our data set, given we don't know the shape of our data? By partition, I refer to partitioning cluster algorithms such as kMeans and kMedoids.
Q3. What mind map would experienced clustering users have when they intend to apply partitioning cluster algorithms?
For example, the following is mine, but I am inexperienced in clustering:
- check if the data is comprised of continuous, or continous + discrete variables
- pick euclidean/manhattan/mahalanobis distance measure for former, or gower for latter
- perhaps standardise first and then apply the distance measure/or correlation if a similarity matrix is sought
- apply a partitioning cluster algorithm; kMeans if we know our data shape and outliers are not a problem. kMedoids if we don't want to be affected by outliers (although the definition of a centroid/prototype as found by kMeans seems to change if we apply kMedoids, i.e. a centroid in kMeans is understandably a prototype of each cluster, while in kMedoids this centroid is simply another data point and not a prototype)
- validate cluster results (using the Elbow test/Dunn index/Silhouette Coeff if focusing only on internal measures) and also check for clustering tendency
Am I missing something important in the above?
Many thanks, and please pardon the ignorance.