I'm working with k-means algorithm, and I'm proceeding in this way:
- I've run k-means from 2 to n clusters, I plotted the k-means result of the variance, to get the "elbow", to decide the best trade-off between model and number of clusters, the best result is 4;
- I repeated k-means n times, let's say 20,000 times, each time with 4 clusters;
- I've stored all the coordinates of the centroids calculated;
- I've calculated the mean coordinates of each centroids, that is a mean based on all the 20,000 iteration;
- With the new centroids, I calculated for each point in my dateset the smallest euclidean distance, to get the "best" centroids for each point.
The problem is that I got my points distributed in only 3 centroids, i.e. the fourth is not given to anyone. This is in contrast also with the point 1, but if I decided to have 5,6, etc. cluster, there is always an cluster not used.
I know that this mean that a centroids is too much away from my points, but is it correct? And my procedure, is it meaningful? Should I calculate the variance to see the goodness of fit of the model?