Use K-means clustering on SVD/PCA of data In an assignment I was suppose to perform K-means clustering on the MNIST dataset (just the 0's and the 1's) and then use SVD/PCA to visualize the data in two dimensions. I missunderstood this and performed the K-means on the SVD of the dataset and was told that this is not something one is supposed to do. However when I look at the result of the K-means clustering on the SVD of the data I get a very similar result (atleast I can't tell the difference from the 2-dim plot). What is the reason why you don't want to do this?
 A: When you use PCA (in your case with two principal components), you lose information that you might need for clustering. However, clustering doesn't remove information that you need for PCA.
Let's say you have the situation as in this figure:

where the blue cluster contains the zeros and the red one the ones. The black line is the PCA subplane you project to. Now, if you first cluster and then project, you get a two-dimensional plot that might look like this:

But if you do PCA first, to get the two-dimensional projection of your data, you would end up with:

and k-means would have a much harder time clustering the data, and maybe come up with something as follows:

You probably have been asked to do PCA only so you could better look at the k-means result (after all, one can only plot in two dimensions).
It is sometimes recommended to do SVD before clustering, but this is then often only either because one wants to reduce memory space and/or computation time (if your data is high-dimensional) or you have some expert knowledge about your specific scenario which suggests projecting first.
In any case, SVD removes information that you could need for clustering.
