Non-linear projection in self organizing maps

Question

I have difficulty understanding how self organizing maps (SOM) are doing dimensionality reduction. Can anybody provide a useful explanation to me?

Suppose we have 20 training data points in 50 dimensions. Let's say, I have specified 3 by 3 SOM (lattice with 9 points), I embed my manifold (3 by 3 lattice) to 50-D space and after the training process, each data point is mapped to one of the 9 points (nodes) in my manifold. Now, my embedded manifold (3 by 3 SOM) are 50-D. So how come I'm back to 2-D dimensions? I mean, where is this non-linear projection?

Answer 1

Each of your 9 SOM grid points has two locations: one in your 50D problem space and one in the 2D SOM lattice space. The SOM algorithm moves them in your problem space, but in a way that preserves their (non-changeable) relationships in 2D space. Look at the SOM algorithm and you'll see this.

The SOM lattice point that is closest to the next training example (in 50D space) is moved towards that example (in 50D space), proportional to its distance (in 50D space). And so are the SOM lattice points that are neighbors (in 2D space) of the winning SOM lattice point, based on their distances (in both 2D and 50D space).

It's the part of the algorithm described in the last sentence that's the key. It organizes the SOM points (in 50D space) as a 2D manifold, which is how you get a projection from 50D to 2D. (The algorithm decreases the 2D space radius of effect over time, which is an important part of keeping things manifold-like in 50D space.)

Answer 2

Your 3 by 3 SOM is approximating a 2D manifold. Your points mapped to a $3 \times 3$ grid is the "projection". If you had more data, and used a larger grid, say of $100 \times 100$, then it may be more intuitive.

If you want to find a 3D manifold, you could use a $3 \times 3 \times 3$ grid. Is it more clear now?