10
$\begingroup$

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used?

I perform LSI in R using the prcomp PCA function and extract the features with highest loadings from the first $k$ components. By that I get the features describing the component best.

I tried to use the kpca function (from the kernlib package) but cannot see how to access the weights of the features to a principal component. Is this possible overall when using kernel methods?

Improve this question
$\endgroup$
1
  • $\begingroup$ Have you tried princomp function? $\endgroup$
    – mariana soffer
    Mar 12, 2011 at 3:04

2 Answers 2

Reset to default
7
$\begingroup$

I think the answer to your question is negative: it is not possible.

Standard PCA can be used for feature selection, because each principal component is a linear combination of original features, and so one can see which original features contribute most to the most prominent principal components, see e.g. here: Using principal component analysis (PCA) for feature selection.

But in kernel PCA each principal component is a linear combination of features in the target space, and for e.g. Gaussian kernel (which is often used) the target space is infinite-dimensional. So the concept of "loadings" does not really make sense for kPCA, and in fact, kernel principal components are computed directly, bypassing computation of principal axes (which for standard PCA are given in R by prcomp$rotation) altogether, thanks to what is known as kernel trick. See e.g. here: Is Kernel PCA with linear kernel equivalent to standard PCA? for more details.

So no, it is not possible. At least there is no easy way.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ (+1) I think it's easier to explain with an analogy to SVMs, where in linear space you can compute the weights each variable contribute to the separating hyperplane (kinda a importance measure, usable for feature selection at least) while in kernel spaces it's too complex or outright impossible to do. Same logic here. $\endgroup$
    – Firebug
    Jun 13, 2016 at 20:19
1
$\begingroup$

The following example (taken from kernlab reference manual) shows you how to access the various components of the kernel PCA:

data(iris)
test <- sample(1:50,20)
kpc <- kpca(~.,data=iris[-test,-5],kernel="rbfdot",kpar=list(sigma=0.2),features=2)

pcv(kpc)        # returns the principal component vectors
eig(kpc)        # returns the eigenvalues
rotated(kpc)    # returns the data projected in the (kernel) pca space
kernelf(kpc)    # returns the kernel used when kpca was performed

Does this answer your question?

Improve this answer
$\endgroup$
4
  • $\begingroup$ i tried rotated(kpca) thinking it is the same as prcomp$rotation; which is (taken form R help(prcomp)): "rotation: the matrix of variable loadings (i.e., a matrix whose columns contain the eigenvectors)." However it is not. The question is however also meant very general because i'm not sure if LSA/LSI is possible using non-linear dimensionality reduction at all. $\endgroup$
    – user3683
    Mar 14, 2011 at 21:47
  • $\begingroup$ sorry, but i might be missing the question; Why do you think that non-linear dimensionality reduction is not possible in LSA/LSI? $\endgroup$
    – Lalas
    Mar 14, 2011 at 22:54
  • 1
    $\begingroup$ because there is no linear combination of the dimensions but one that depends on a kernel function. Is it in this setting possible to determine the weight one dimension has for a (non-linear) principal component? $\endgroup$
    – user3683
    Mar 29, 2011 at 6:24
  • 2
    $\begingroup$ (-1) This might be a useful code snippet, but I don't think it answers the original question at all. $\endgroup$
    – amoeba
    Sep 19, 2014 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.