1
$\begingroup$

I have a few gaps in my understanding:

I know that PCA does dimensionality reduction. It does so by finding transformations such that the projection of the data points onto certain lines minimize the error of such projection (the Frobenius norm). PCA finds these lines, and these lines are linear combinations of existing variables.

Manifold hypothesis says that data must be intrinsically low dimensional.

My question is: since, PCA can only find linear transformations, so PCA can't do anything if the manifold is non-linear. Is any modification of PCA possible to extend it to find non linear manifolds?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction offers a list of 26+ methods. $\endgroup$
    – amoeba
    Commented May 1, 2017 at 21:11
  • $\begingroup$ @amoeba thanks, also, one more thing, I read that PCA minimizes the error of projection, but nowhere in the PCA method there is an explicit step doing so. I understand that the step of projection is implicit in the matrix multiplication, but the error term is not. Could you point to any resources that explain that or any pca extensions that modify that error itself. thanks. $\endgroup$
    – Rafael
    Commented May 1, 2017 at 21:24
  • 1
    $\begingroup$ Are you asking why PCA minimizes the projection error? You can start by reading my answer here stats.stackexchange.com/questions/2691 and then follow the links if you like. $\endgroup$
    – amoeba
    Commented May 1, 2017 at 21:28
  • 1
    $\begingroup$ cirano.qc.ca/pdf/publication/2004s-27.pdf is an article on nonlinear dimensionality reduction using spectral (eigenvalue) methods $\endgroup$
    – Jakub Bartczuk
    Commented Sep 6, 2017 at 13:07

1 Answer 1

Reset to default
3
$\begingroup$

Kernel PCA might be helpful. It performs linear PCA of the data projected in a reproducing kernel hilbert space where the data almost always lies on a linear manifold

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.