In "Pattern Recognition and Machine Learning" by Bishop, the following problem about Kernel PCA is laid out : In linear PCA, we can approximate data points by projecting them onto the $L < D$-dimensional subspace, defined by:

$ \hat{x_n} = \sum_{i=1}^{L} (x_n^T u_i) u_i$

where $u_i$ are the Eigenvectors. From my understanding this is used for de-noising, for example.

Bishop then explains that this is not possible for kernel PCA, since it is not guarented that a vector the pre-image $x$ of $\phi(x)$ lies on the nonlinear D-dimensional manifold. In particular, he writes:

In kernel PCA, this [projection] will in general not be possible. To see this, note that the mapping $\phi(x)$ maps the $D$-dimensional $x$ space into a $D$-dimensional manifold in the $M$-dimensional feature space $\phi$. [...] However, the projection of points in feature space onto the linear PCA subspace in that space will typically not lie on the nonlinear $D$-dimensional manifold and so will not have a corresponding pre-image in data space.

What nonlinear $D$-dimensional manifold is meant here? From my understanding, $\phi(.)$ maps from the $D$ dimensional space to the feature space. Thus every image $\phi$ has, by definition, a pre-image in the $D$ dimensional space.

What am I missing here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.