2
$\begingroup$

Suppose we have 20 training data points in 50 dimensions. Let's say I have specified a 3 by 3 SOM (lattice with 9 points), I embed my manifold (3 by 3 lattice) to 50-D space and after training each data point is mapped to one of the 9 points (nodes) in my manifold. Now, my embedded manifold (3 by 3 SOM) is 50-D. So how do I come back to 2-D? In other words, where is this non-linear projection?

$\endgroup$
3
$\begingroup$

SOMs are not a method of dimensionality reduction in the same sense that PCA is. This is a method of visualizing some structure of the high dimensional data, but it does not build an actual mapping $\phi:X\rightarrow\mathbb{R}^2$. Instead, it creates a map (when using 2dimensional grid) $\phi:X\rightarrow L \subset \mathbb{R}^2$, where $L$ is finite set of predefined points (neurons) with some "topology" (constraints). In your case, it builds $\phi : X \rightarrow \{-1,0,1\} \times \{ -1,0,1\}$ ($3 \times 3$ lattice in $\mathbb{R}^2$), which is defined as $\phi(x) = {\arg\min}_{n \in SOM}\left \| x-f(n) \right \|$, where $SOM$ is a set of neurons (in your lattice) and $f$ is a function returning the coordinates of the given neuron in the input space.

There are at least few ways of visualizing the trained SOM:

  • one can render the SOM's neurons as points in the input space, with edges connecting the topologically close ones (this is possible only if the input space has small number of dimensions, like 2-3, so this is not applicable in your case)
  • display data classes on the SOM's topology - if your data are labeled with some numbers $\{1,..k\}$, we can bind $k$ colors to them, for the binary case let us consider blue and red. Next, for each data point we calculate its corresponding neuron in the SOM and add this label's color to the neuron. Once all data have been processed, we plot the SOM's neurons, each with its original position in the topology, with the color being some aggregate (e.g., mean) of colors assigned to it. This approach, if we use some simple topology like 2d grid, gives us a nice low-dimensional representation of data. In the following image, subimages from the third one to the end are the results of such visualization, where the red color means label 1 ("yes" answer) and the blue means label 2 ("no" answer). This shows a distribution of classes and can serve as a preprocessing method for further analysis.
  • one can also visualize the inter-neuron distances by calculating how far away are each connected neuron is and plotting it on the SOM's map (second subimage in the visualization) - this shows the shape of the lattice somewhat.
  • one can cluster the neuron's positions with some clustering algorithm (like K-means) and visualize the clusters ids as colors (first subimage), which is an even simpler shape visualization then the previous one.

Image from wikipedia

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.