1
$\begingroup$

I need to perform kernel PCA on the colon-­‐cancer dataset and then I need to plot number of principal components vs classification accuracy with PCA data.

For the first part I am using kernlab in R as follows (let number of features be 2 and then I will vary it from say 2-100):

kpc <- kpca(~.,data=data[,-1],kernel="rbfdot",kpar=list(sigma=0.2),features=2)

I am having tough time to understand how to use this PCA data for classification (I can use any classifier, e.g. SVM). Basically my question is how to use kernel PCA for classification?

Data look like this:

colon cancer cleaned data

Uncleaned original data look like this:

colon cancer Uncleaned data

$\endgroup$

2 Answers 2

3
$\begingroup$

PCA and its kernelized version are data transformations, typically employed for dimensionality reduction or denoising. They are not used for classification directly, though you could use (standard) PCA as a preprocessing step before training a classifier.

The most common kernel-based classification methods are support vector machines (extremely popular), kernel LDA (less popular) and kernel logistic regression, so you probably want to read up on those.

$\endgroup$
3
  • 1
    $\begingroup$ +1, but perhaps OP is trying (or is told) to use kernel PCA for nonlinear transformation / dimensionality reduction + linear SVM for classification afterwards. Is it a reasonable strategy, or would you rather (as I suspect) recommend to use kernel SVM directly? $\endgroup$
    – amoeba
    Mar 19, 2015 at 20:54
  • 2
    $\begingroup$ @amoeba Indeed, I would go for kernel SVM directly. I can't think of any immediate advantage of a two-step approach compared to kernel SVM. $\endgroup$ Mar 19, 2015 at 21:06
  • $\begingroup$ yes u all are right but i wanted to plot number of principal components vs classification accuracy. $\endgroup$ Mar 20, 2015 at 3:34
0
$\begingroup$

From what I understand, the PCA is used to reduce the dimension of your problem. Example: Let's say you have 10000 features to train a Neural Network. This would take too much time to train. So you use Princial Component Analysis to find relations between the variables and you reduce the feature number to 1000 (which now they are not exactly the same features as before, but a projection in other plane). In this process you lose some information and you can calculate this using the variance explained here

Hope this helped! Tiago

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.