How to project high dimensional space into a two-dimensional plane? I have a set of data points in a N-dimensional space. In addition, I also have a centroid in this same N-dimensional space. Are there any approaches that can allow me to project these data points into a two-dimensional space while keeping their relative distance information in the original space. Is PCA the correct one?
 A: A general framework which addresses your problem is called dimensionality reduction. You would like to project data from N dimensions to 2 dimensions, while preserving the "essential information" in your data. The most suitable method depends on the distribution of your data, i.e. the N-dimensional manifold. PCA will fit a plane using least squares criterion. This will probably work poorly for the "swiss roll" example: swiss roll.
More modern methods include Kernel PCA, LLE, diffusion maps and sparse dictionary representations. Regarding distance preservation, some methods can preserve non-euclidean distances.
A: As mentioned in the previous answer, there are a number of methods of dimensionality reduction, and an important thing to consider is what are you trying to represent - are you interested in Euclidean distance measures? Or a metric of similarity across samples?
For the former, PCA can be appropriate. It is commonly used with continuous measures such as measurements of samples (animals, plants, etc...). I would look into the more modern mentions in the earlier answer too though.
For the latter, where you might be trying to compare similarity using a non-euclidean distance metric, a few good methods exist such as Principle Components Ordination (PCoA) and Non-metric Multidimensional Scaling (NMDS). An example of when you might use these is when you are comparing the ecological communities among different areas, and you have numbers of different types of organisms that were found. So, your data are "count" data. There are a number of similarity metrics such as Jaccard, Sorensen, Bray-Curtis, that effectively let you estimate how similar the sites are in their composition of organisms. PCoA and NMDS basically let you to plot the samples (sites) to represent the ecological distance (similarity), and you have a score for site on each axis. 
There are lots of good books and other resources for multivariate analysis. Search for "Ordination" on Google. Also, there's an R package called 'vegan' that is really good for actually carrying out a lot of this work.
A: Your problem sounds like a text-book application for multidimensional scaling. A good introduction can be found here: http://www.mathpsyc.uni-bonn.de/doc/delbeke/delbeke.htm
Of course you can try PCA. But PCA has no intention to keep the relative distance information in the original space.
