3
$\begingroup$

Background

I have a set of data points in high-dimensional (512D) space that I wish to map to 2D for visualisation. I am interested in observing in 2D the (approximate) relative distances between the data points and their general spatial structure.

Currently I am using multidimensional scaling within Matlab to do this:

% Dissimilarity matrix
D = pdist(X', 'euclidean');
% Non-metric MDS -- force dimensionality to 2D
Y = mdscale(D, 2)';
% Display result
figure, plot(Y(1, :), Y(2, :), 'o');

Where X is the set of high-dimensional data points (512 x number of points).

Having performed this mapping for a set of points, I wish to be able to project/map new high-dimensional points within the 2D space without needing to use the original set of high-dimensional points.

Questions

(1) How do I obtain the mapping/projection used to scale the 512D points to 2D so that I can apply it to new data points?

(2) What would be a suitable "error metric" to evaluate the projections? The purpose of doing this is to compare different dimensionality reduction techniques (e.g. MDS vs LLE).

For example, would a comparison of the distances between respective k nearest neighbours within both spaces be suitable? i.e. sum the squared differences between corresponding neighbour distances within the two spaces for each point and compute the mean.

Improve this question
$\endgroup$
3
  • $\begingroup$ so that I can apply it to new data points? How do you see it? MDS input is one or several matrices of a dissimilarity within a fixed set of objects. $\endgroup$
    – ttnphns
    Feb 28, 2013 at 5:03
  • $\begingroup$ So I am unable to obtain a projection since the point location information is lost by the distance (dissimilarity) matrix? What about comparing the projections of multiple dimensionality reduction techniques? Can I evaluate their relative effectiveness by comparing the size of the differences in distance between neighbouring points with respect to both spaces? Does this make sense, or is there a better way of doing or looking at this? $\endgroup$
    – Josh
    Feb 28, 2013 at 7:04
  • 1
    $\begingroup$ It's not because something is "lost" but because MDS is not a "forcasting" method to be able to find coordinates for new points on an old map. Consider using PCA or Correspondense analysis which can do it. $\endgroup$
    – ttnphns
    Feb 28, 2013 at 13:08

1 Answer 1

Reset to default
1
$\begingroup$

A toolbox can be a cage. If you are doing something new it can be instructive to write it without the toolbox the first time.

The points in the 512 (or whatever) dimensional space might be able to be mapped to any subset of it. Without knowing something about the data, it could throw away all but 1/512th of the information to map to a 2d space.

So not all spaces are created equal. Some are planes oriented with the axes (stunningly convenient but equally rare) but others are surfaces of spheres, or follow differential equations. Some may have projections to 2d surfaces for which we currently have no formalism, but future developments open them up to approach.

If you do not have a clean way to do the projection, you are going to have to throw away information.

The quick way, as @ttnphns suggested, to determine if you have plane-ish clouds of data, and throw away the non-informative dimensions perpendicular to those planes, is PCA (principle component analysis).

Here are some links:

Best of luck.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.