linear versus nonlinear dimensionality reduction techniques I was going through a short tutorial on dimensionality reduction techniques. Some of these techniques are linear while others are non-linear. What is the distinction between them? Why the terms 'linear' and 'non-linear'?
 A: Linear vs. non-linear are two different types of transformations. Here gives the details of the linear transformation.

A linear transformation between two vector spaces $V$ and $W$ is a map $T:V->W$ such that the following hold:

*

*$T(v_1+v_2)=T(v_1)+T(v_2)$ for any vectors $v_1$ and $v_2$ in $V$, and

*$T(\alpha v)= \alpha T(v)$ for any scalar alpha.


For example, in dimension reduction domain, principal component analysis (PCA) is a linear transformation. And kernel PCA is a non-linear one.
Here are details (thanks @whuber for the suggestion).
Suppose we have data matrix $X$ and the Singular-value decomposition of $X$ is $X=UDV^T$, the PCA transformation is just a matrix multiplication on $X$, which is $XV$.
We can show the matrix multiplication is linear algebraically: Define $T(X)=XV$ (the transformation of $X$ is multiply $X$ by matrix $V$), then we know $(X_1+X_2)V=X_1V+X_2V$ and $\alpha X V=\alpha XV$.
Intrusively, you can think about linear transformation is shifting and stretching the data, and non-linear transformation will make more dramatic changes on data such as making data "inside out".
If we visualize the transformation, linear transformation looks like this

Nonlinear transformation looks like this

Picture source
http://mathinsight.org/media/image/source/linear_transformation_2d_m1_m1_1_3.svg
http://2.bp.blogspot.com/_slrAR0IXTL0/TF-OZaNbRCI/AAAAAAAAAUo/SdYS3hXd4MI/s1600/figure.png
