I'm comparing the Support Vector Machines (SVM) formulation of linear PCA with kernel PCA. I know that in linear PCA, the maximum number of principal components is equal to the dimension of the input space. For kernel PCA, therefore, I expect the maximum number of principal components to be equal to the dimension of the projection of the input space into higher dimensional space (which is often implicit by just using the Kernel function). I don't find any confirmation on this somewhere online. Moreover, in my notes of a lecture I've followed, I've noted somewhere that the maximal number of principal components of kernel PCA is the number of observations in the dataset. Does anyone have a formal understanding of how this works or can advice a resource online? Does the number of principal components depend on the dual/primal formulation? Thanks a lot for some advice on this!
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
I believe that the matrix in the eigenvalue-eigenvector problem is the kernel matrix, that is a NxN matrix (N is the number of examples). So there will be N eigenvalues and N eigenvectors. Each eigenvector is a principal component and each eigenvalue is the explained variance associated with that component. So in kernel PCA you can have at most N dimensions.
Mathematically, there is more to it but I think this is the general gist.
-
$\begingroup$ agree. maximum number of final components is the rank of the matrix of X in kernel space, which has dimension nxm, where m could be as large as $\infty$. so it depends, but is never more than n. $\endgroup$– carloCommented Oct 18, 2019 at 9:06