1
$\begingroup$

According to this website, kernel PCA with RBF (Gaussian) kernel can separate half-moon shapes and concentric circles effectively but not Swiss Roll shapes (in 3-D).

I don't understand why it doesn't work with Swiss Roll and how the point in 3-D is actually mapped to a point in a higher dimension. The article stated that

a (Gaussian) radial basis function (RBF) kernel can be used to map the data onto infinite dimensions

but I don't understand it. What are these "infinite dimensions"?

Also, can you give me an intuitive guideline in which distribution of the cluster of data I should apply RBF and in which cases I should avoid using it?

$\endgroup$
2
$\begingroup$

It should work with the swiss roll data set.

The problem of RBF is the need to choose the bandwidth parameter. It is also computationally rather expensive and tends to overfit much more. It is related to nearest-neighbor classification (with exponential weights).

The vector space it maps to is hard to grasp. Roughly, every possible point in space (infinitely many!) is a dimension, and the coordinate value is the exponential of the distance. Close points will have similar values in each dimension.

$\endgroup$
  • $\begingroup$ Could you say a little more about what you mean by "It should work with the swiss roll data set"? Empirically, I've never seen kernel PCA be able to unroll the swiss roll when used with a vanilla, isotropic RBF kernel (for any choice of bandwidth). But, I could be missing something, and it would be interesting if it does work in some regime. It must work with some kernel because isomap (which can unroll the swiss roll) can be formulated as kernel PCA with a particular kernel choice. Were you referring to RBF kernels in particular? Isotropic? $\endgroup$ – user20160 Mar 11 '18 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.