# How to compare PCA with KPCA for dimension reduction?

Both linear principal component analysis (PCA) and kernel principal component analysis (KPCA) are unsupervised dimension reduction methods. I have a dataset with $4000$ training samples and $40000$ test samples. The dimension of their features is $200$. To fit the i.i.d assumption of PCA and KPCA, I randomly select a number of anchor samples from both training set and test set, to conduct PCA and KPCA. Suppose it is 5000 and use Gaussian kernel as the kernel function of KPCA. The $\sigma$ is selected from the range $[10:10:100,200:100,500]$.

Let their reduced dimensions $d$ be selected from the range $[10,20,...,100]$. Then based on the reduced features, I conduct a RBF-SVM classification. The hyper-parameters of RBF is selected by 5-fold cross-validation.

From the classification results, it seems to be no significant different between PCA and KPCA. But in common sense, KPCA should be better than PCA. Are there any possible explanations?

• I didn't quite understand the title. What is dimension dimension? – ttnphns Jan 27 '16 at 9:22
• I guess it should read dimension reduction? – Vincent Guillemot Jan 27 '16 at 9:27
• Did you also compare the classification results with the situation where there is no dimension reduction? – Vincent Guillemot Jan 27 '16 at 9:37
• @ttnphns, I have revised it. Please kindly check it. Thanks. – mining Jan 27 '16 at 9:56
• @VincentGuillemot, thanks! The result without dimension reduction seems to be a little better than that with dimension reduction, say about 0.1%. – mining Jan 27 '16 at 9:57