4
$\begingroup$

I have a set of 500 points in 5D. Each point belongs to one of five classes, and the class labels are known.

I’d like to visualise the dataset in 2D such that the classes would be separated as much as possible.

I am currently using PCA and doing a scatterplot of the first two principal components. This works quite well for some datasets, but not as well for others. This makes intuitive sense, since PCA maximises explained variance rather than separability.

Are there any known methods for finding a 2D projection that would maximise separability? I don’t have any specific measure in mind and am open to suggestions.

(Tagging with [r] as I'd love to see some R code or pointers.)

Improve this question
$\endgroup$
  • 2
    $\begingroup$ Perhaps multiclass linear discriminant analysis (LDA)? $\endgroup$ – purple51 Oct 15 '14 at 10:21
9
$\begingroup$

You may want to try linear discriminant analysis (LDA). The basic idea of LDA is to project your data in a space where the variance within classes is minimum and the one between classes is maximum.

Unlike PCA, LDA uses the labels to reduce the dimensionality. Indeed, discovering the direction that maximizes the variance with PCA does not mean projecting in this direction will help you to classify.
Take the example of classification of pictures. The variance of the average light may vary a lot between your images, yet that feature might not help you that much depending on your goal.

See this post on how to do it with R.

Improve this answer
$\endgroup$
2
$\begingroup$

You're looking for multi-dimensional scaling (MDS). It's purpose is to visualize high-dimensional data by mapping into a lower dimensional space (usually 2D) while trying to retain distance information between points.

In your case, using just class equality as distance would result in the same projection for all points within the same class. Maybe you can try a linear combination of euclidean distance and class equality as used distance metric.

See the Wikipedia article for further details.

Improve this answer
$\endgroup$
  • $\begingroup$ From a quick scan of the Wikipedia article, this sounds exactly what I am looking for. I'll read up further. Thanks! $\endgroup$ – NPE Oct 15 '14 at 11:41
  • $\begingroup$ @NPE Now that I think about it again, it doesn't quite fit your case completely. If you only take class equality as distance, all projected points from the same class would coincide. Maybe you need to use a combination of class and (euclidean) distance of your 5D points. $\endgroup$ – ziggystar Oct 15 '14 at 13:44
  • $\begingroup$ If you just use class equality, you're effectively throwing all the data away. Your projection will be the trivial visualization of the fact that classes are different a priori. OP, you are looking for Neighborhood Component Analysis: en.wikipedia.org/wiki/Neighbourhood_components_analysis The objective function of NCA is to project the data into a space where Nearest Neighbor classification is more accurate. This is exactly what you want. $\endgroup$ – user1134516 Oct 15 '14 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.