If I do a transform and then an inverse_transform using PCA, I of course get the original data back. If I do the same for KernelPCA, I don't. Is this a property of kernelPCA or a shortcoming of the implementation? Code is here:

import numpy as np
from sklearn.decomposition import PCA, KernelPCA
pca = PCA(n_components=2, copy=True)
kpca = KernelPCA(n_components=5,
             gamma=1.0, # default 1/n_features
train_set = np.random.rand(5,2)
k_transformed = kpca.fit_transform(train_set)
k_orig =  kpca.inverse_transform(k_transformed)
p_transformed = pca.fit_transform(train_set)
p_orig =  pca.inverse_transform(p_transformed)
print "Original Data"
print train_set
print "PCA"
print p_orig
print "KPCA"
print k_orig

Output is here:

Original Data
[[ 0.60102465  0.37562677]
 [ 0.78281304  0.20575771]
 [ 0.55120131  0.31717359]
 [ 0.48216065  0.85297703]
 [ 0.77400554  0.86559728]]
[[ 0.60102465  0.37562677]
 [ 0.78281304  0.20575771]
 [ 0.55120131  0.31717359]
 [ 0.48216065  0.85297703]
 [ 0.77400554  0.86559728]]
[[ 0.53530411  0.3594765 ]
 [ 0.51250412  0.25227312]
 [ 0.51253417  0.32343322]
 [ 0.45231888  0.55243246]
 [ 0.49755706  0.55221213]]

From wikipedia - " Because we are never working directly in the feature space, the kernel-formulation of PCA is restricted in that it computes not the principal components themselves, but the projections of our data onto those components. " . So kernel PCA only computes the final step which is the projection onto the a subspace. There is never a unique way to recover a vector from it's projection.

| cite | improve this answer | |
  • $\begingroup$ If that is true, then what does the function inverse_transform do ? $\endgroup$ – TPM Oct 2 '17 at 13:24
  • $\begingroup$ For a function to have an inverse, it has to be 1-1. Non-trivial Projections are never 1-1. Suppose you have two variables x and y so the (x,y) identifies your data and suppose that x accounts for 96% of it. If you are only given x, one can't determine what y is. In PCA, one starts with (x,y) and one is told that x has enough of the variance. But one has (x,y) and the matrix (matrices) that gives the projection onto y. With kernel PCA one merely knows that there is a non-linear embedding $R^2 \hookrightarrow R^N$ and one is given $f_1(x,y) $, one doesn't have $f_1, \ldots f_N$ . $\endgroup$ – meh Oct 2 '17 at 13:44
  • $\begingroup$ In the PCA example (to use your notation) I used both x and y, so the inverse_transform was exact. My expectation is that if I use all M KernelPCA components (where M is the number of data points, not the number of dimensions), then the inverse transform ought to be exact, just as when I use all N (number of dimensions) components in the linear PCA. Is that a wrong assumption? $\endgroup$ – TPM Oct 3 '17 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.