To find the most important variable for each Principal Component is easy with PCA:

With data->X and variables->variable_names

important=[np.abs(pca.components_[i]).argmax()for i in range(n)]
important_names = [variable_names[important[i]] for i in range(n)]
important_variables= {'PC{}'.format(i+1): important_names[i] for i in range(n)}
important_variables_f= pd.DataFrame(important_variables.items())

However, how can I accomplish the same result with Kernel PCA, since it has no components_ attribute?


With PCA, you rely on the magnitude of the coefficient in the principal component vector because each of the coefficients of the PC is explicitly multiplied with features while calculating the projection. However, in Kernel PCA, the transformed vectors, $\phi(x)$, are never explicitly calculated; we directly get the projections. Especially, in some cases like Gaussian kernel, these vectors are infinite dimensional, and can't be stored. For finite mappings, you could try to rewrite the exact mapping and perform PCA directly on the transformed data to get which of the new features are important (based on this variance based criterion).

  • $\begingroup$ Thank you for your answer! I was a little bit confused since the attribute alphas_ of Kernel PCA should give me the eigenvectors, but compared to the components_ attribute from PCA the length of the vectors don't fit with the number of variables. So you mean I should basically map my data, then apply PCA and at last find the most important variables the same way as I did with normal PCA? $\endgroup$ – M08 Aug 4 at 20:35
  • $\begingroup$ Yes, and you can do that with a finite-length mapping. Also, note that the important variables will be the transformed ones, not the older ones. $\endgroup$ – gunes Aug 4 at 20:37
  • $\begingroup$ Which kernel would you use and Is it then possible to retrace the transformed variables to the originals? $\endgroup$ – M08 Aug 4 at 20:41
  • $\begingroup$ For example, polynomial kernels can be easy to use and trace. $\endgroup$ – gunes Aug 4 at 20:41
  • $\begingroup$ perfect, thank you very much! $\endgroup$ – M08 Aug 4 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.