# Is it possible to use kernel PCA for feature selection?

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used?

I perform LSI in R using the prcomp PCA function and extract the features with highest loadings from the first $k$ components. By that I get the features describing the component best.

I tried to use the kpca function (from the kernlib package) but cannot see how to access the weights of the features to a principal component. Is this possible overall when using kernel methods?

• Have you tried princomp function? – mariana soffer Mar 12 '11 at 3:04

But in kernel PCA each principal component is a linear combination of features in the target space, and for e.g. Gaussian kernel (which is often used) the target space is infinite-dimensional. So the concept of "loadings" does not really make sense for kPCA, and in fact, kernel principal components are computed directly, bypassing computation of principal axes (which for standard PCA are given in R by prcomp$rotation) altogether, thanks to what is known as kernel trick. See e.g. here: Is Kernel PCA with linear kernel equivalent to standard PCA? for more details. So no, it is not possible. At least there is no easy way. • (+1) I think it's easier to explain with an analogy to SVMs, where in linear space you can compute the weights each variable contribute to the separating hyperplane (kinda a importance measure, usable for feature selection at least) while in kernel spaces it's too complex or outright impossible to do. Same logic here. – Firebug Jun 13 '16 at 20:19 The following example (taken from kernlab reference manual) shows you how to access the various components of the kernel PCA: data(iris) test <- sample(1:50,20) kpc <- kpca(~.,data=iris[-test,-5],kernel="rbfdot",kpar=list(sigma=0.2),features=2) pcv(kpc) # returns the principal component vectors eig(kpc) # returns the eigenvalues rotated(kpc) # returns the data projected in the (kernel) pca space kernelf(kpc) # returns the kernel used when kpca was performed  Does this answer your question? • i tried rotated(kpca) thinking it is the same as prcomp$rotation; which is (taken form R help(prcomp)): "rotation: the matrix of variable loadings (i.e., a matrix whose columns contain the eigenvectors)." However it is not. The question is however also meant very general because i'm not sure if LSA/LSI is possible using non-linear dimensionality reduction at all. – user3683 Mar 14 '11 at 21:47