I know that the cluster centroid is the middle of a cluster. It's a vector containing one number for each variable, where each number is the mean of a variable for the observations in that cluster.
I cluster my dataset (MNIST handwritten digits) using K-Means into 3,5 and 10 clusters. My question is: which characteristic of the data is captured by the centroids?
Plotting the centroids as images, I can see that with 3 clusters centroids are not well defined. For example, the 3 and 7 digits overlap, as you can see in the image. The same thing happens with 4 and 5 digits. While with 10 clusters, centroids are better defined (as you can see in the image), but there are some repetitions for certain values (eg 2 centroids for 3 and 4). Why is this happening?