I have a set of data points in a N-dimensional space. In addition, I also have a centroid in this same N-dimensional space. Are there any approaches that can allow me to project these data points into a two-dimensional space while keeping their relative distance information in the original space. Is PCA the correct one?
$\begingroup$
$\endgroup$
3
asked Jul 7, 2013 at 21:27
-
1$\begingroup$ If you want to try to preserve distances, my first thought would have been multidimensional scaling on the distances themselves (which is related to PCA), but since you have the locations and not just the distances, by my understanding, PCA should work for that. $\endgroup$– Glen_bJul 7, 2013 at 23:14
-
1$\begingroup$ @Glen_b, The key point is not that MDS is for distances input and PCA is for coordinates input, but that iterative MDS fits few dimensions while PCA retains few dimensions. So MDS preserves distances somewhat better than classic PCA does. The answer for the question is Yes, PCA is suitable, but MDS is more suitable. $\endgroup$– ttnphnsJul 8, 2013 at 8:14
-
1$\begingroup$ This largely what is studied in the field of metric space embedding, i.e. how can you reduce the dimensionality of your data while minimizing distortion of distances. $\endgroup$– BitwiseJul 13, 2013 at 23:48
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
A general framework which addresses your problem is called dimensionality reduction. You would like to project data from N dimensions to 2 dimensions, while preserving the "essential information" in your data. The most suitable method depends on the distribution of your data, i.e. the N-dimensional manifold. PCA will fit a plane using least squares criterion. This will probably work poorly for the "swiss roll" example: swiss roll.
More modern methods include Kernel PCA, LLE, diffusion maps and sparse dictionary representations. Regarding distance preservation, some methods can preserve non-euclidean distances.
-
4$\begingroup$ It is important to note that "dimensionality reduction" methods typically do not maintain "relative distance information." Whether they do or not depends partly on the method and partly on the intended "distance." $\endgroup$– whuber ♦Jul 8, 2013 at 12:52
$\begingroup$
$\endgroup$
As mentioned in the previous answer, there are a number of methods of dimensionality reduction, and an important thing to consider is what are you trying to represent - are you interested in Euclidean distance measures? Or a metric of similarity across samples?
For the former, PCA can be appropriate. It is commonly used with continuous measures such as measurements of samples (animals, plants, etc...). I would look into the more modern mentions in the earlier answer too though.
For the latter, where you might be trying to compare similarity using a non-euclidean distance metric, a few good methods exist such as Principle Components Ordination (PCoA) and Non-metric Multidimensional Scaling (NMDS). An example of when you might use these is when you are comparing the ecological communities among different areas, and you have numbers of different types of organisms that were found. So, your data are "count" data. There are a number of similarity metrics such as Jaccard, Sorensen, Bray-Curtis, that effectively let you estimate how similar the sites are in their composition of organisms. PCoA and NMDS basically let you to plot the samples (sites) to represent the ecological distance (similarity), and you have a score for site on each axis.
There are lots of good books and other resources for multivariate analysis. Search for "Ordination" on Google. Also, there's an R package called 'vegan' that is really good for actually carrying out a lot of this work.
$\begingroup$
$\endgroup$
Your problem sounds like a text-book application for multidimensional scaling. A good introduction can be found here: http://www.mathpsyc.uni-bonn.de/doc/delbeke/delbeke.htm
Of course you can try PCA. But PCA has no intention to keep the relative distance information in the original space.
