Dimensionality reduction (PCA) for plotting text documents on a graph 
*

*I have 50 text documents

*There are 500 possible words, after a stop list has been applied

*My term/document sparse matrix is therefore 50x500


I'd like to cluster these documents. One easy way to do this would be via k-means but that requires giving each document co-ordinates on a 2-d graph.
I've heard that I can reduce the 500-word long vectors per document using dimensionality reduction, specifically PCA. 
Is it realistic that I could just reduce my 50x500 matrix to a 50x2 table and plot those values?
 A: K-means doesn't have a limitation of two features. Actually you should work in the original space to get better results.
If you just want to visualise your dataset indeed you can go for PCA or many of other similar methods.
I prefer classic multidimensional scaling (cmdscale in R). It requires first computing the similarities of your documents in a NxN matrix. Then it is straight forward projecting the documents on a 2D plane with respect to their relative distances.  
A: I would suggest to do some cross-validation. PCA is generally a commonly and successfully used technique for dimensionality reduction, but it also depends on which lower-dimensional space gives you a good classification rate. from 500 down to 2 dimensions sounds like a large step. It might work well, but it is not guaranteed. I would at least do a comparison between 2, 50, 100, 250, and the complete 500 features if in terms of the classification accuracy on the test data set (or cross-validation) afterwards.
Have a look at this article, for example: http://www.hi.info.mie-u.ac.jp/publication/archive/busagala_Camera-ready.pdf

