32
$\begingroup$

As far as I can tell, Kohonen-style SOMs had a peak back around 2005 and haven't seen as much favor recently. I haven't found any paper that says that SOMs have been subsumed by another method, or proven equivalent to something else (at higher dimensions, anyhow). But it seems like tSNE and other methods get a lot more ink now-a-days, for example in Wikipedia, or in SciKit Learn, and SOM is mentioned more as a historical method.

(Actually, a Wikipedia article seems to indicate that SOMs continue to have certain advantages over competitors, but it's also the shortest entry in the list. EDIT: Per gung's request, one of the articles that I'm thinking about is: Nonlinear Dimensionality Reduction. Note that SOM has less written about it than the other methods. I can't find the article that mentioned an advantage that SOMs seem to retain over most other methods.)

Any insights? Someone else asked why SOMs are not being used, and got references from a while ago, and I have found proceedings from SOM conferences, but was wondering if the rise of SVMs or tSNE, et al, just eclipsed SOMs in pop machine learning.

EDIT 2: By pure coincidence, I was just reading a 2008 survey on nonlinear dimensionality reduction this evening, and for examples it mentions only: Isomap (2000), locally linear embedding (LLE) (2000), Hessian LLE (2003), Laplacian eigenmaps (2003), and semidefinite embedding (SDE) (2004).

$\endgroup$
  • 3
    $\begingroup$ Can you link to any of the resources you are referring to? (Eg, which Wikipedia article "seems to indicate..."?) $\endgroup$ – gung Oct 19 '15 at 2:38
  • 11
    $\begingroup$ They seem to have fallen out of favor to an extent that I do not know what SOM refers to. $\endgroup$ – Matthew Drury Oct 19 '15 at 2:54
  • 5
    $\begingroup$ apparently, self-organizing map $\endgroup$ – Christoph Hanck Oct 19 '15 at 7:00
  • $\begingroup$ SOM is just a variant of multidimensional scaling (MDS) which is much older. $\endgroup$ – kjetil b halvorsen Oct 20 '15 at 7:37
  • $\begingroup$ @kjetilbhalvorsen: Do you have any references about SOM and MDS? As I understand it, MDS is global in nature (related to PCA), while SOM is local in nature. Or maybe I misunderstand them. $\endgroup$ – Wayne Oct 20 '15 at 13:29
18
$\begingroup$

I think you are on to something by noting the influence of what the machine learning currently touts as the 'best' algorithms for dimensionality reduction. While t-SNE has shown its efficacy in competitions, such as the Merck Viz Challenge, I personally have had success implementing SOM for both feature extraction and binary classification. While there are certainly some who dismiss SOMs without justification besides the algorithm's age (check out this discussion, there are also a number of articles that have been published within the last few years that implemented SOMs and achieved positive results (see Mortazavi et al., 2013; Frenkel et al., 2013 for instance). A Google Scholar search will reveal that SOMs are still utilized within a number of application domains. As a general rule, however, the best algorithm for a particular task is exactly that - the best algorithm for a particular task. Where a random forest may have worked well for a particular binary classification task, it may perform horribly on another. The same applies to clustering, regression, and optimization tasks. This phenomenon is tied to the No Free Lunch Theorem, but that is a topic for another discussion. In sum, if SOM works best for you on a particular task, that is the algorithm you should use for that task, regardless of what's popular.

$\endgroup$
6
$\begingroup$

I have done research on comparing SOMs with t-SNE and more and also proposed an improvement on SOM that takes it to a new level of efficiency. Please check it out here and let me know your feedback. Would love to get some idea on what people think about it and if it is worth publishing in python for people to use.

IEEE link to paper: http://ieeexplore.ieee.org/document/6178802/

Matlab implementation. https://www.mathworks.com/matlabcentral/fileexchange/35538-cluster-reinforcement--cr--phase

Thanks for your feedback.

$\endgroup$
  • 4
    $\begingroup$ Welcome to Cross Validated! It'd help to give a brief summary of your findings, & the improvement you made, & also perhaps to address the question more directly. $\endgroup$ – Scortchi Apr 5 '17 at 20:29
1
$\begingroup$

My subjective view is that SOMs are less well known and perceived as being less 'sexy' than many other methods, but are still highly relevant for certain classes of problems. It may well be the case that they would have a significant contribution to make if they were more widely used. They are invaluable in the early stages of exploratory data science for getting a feel for the 'landscape' or 'topology' of multivariate data.

The development of libraries such as Somoclu, and research such as that by Guénaël Cabanes (among many others) shows that SOMs are still relevant.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.