Both linear principal component analysis (PCA) and kernel principal component analysis (KPCA) are unsupervised dimension reduction methods. I have a dataset with $4000$ training samples and $40000$ test samples. The dimension of their features is $200$. To fit the i.i.d assumption of PCA and KPCA, I randomly select a number of anchor samples from both training set and test set, to conduct PCA and KPCA. Suppose it is 5000 and use Gaussian kernel as the kernel function of KPCA. The $\sigma$ is selected from the range $[10:10:100,200:100,500]$.
Let their reduced dimensions $d$ be selected from the range $[10,20,...,100]$. Then based on the reduced features, I conduct a RBF-SVM classification. The hyper-parameters of RBF is selected by 5-fold cross-validation.
From the classification results, it seems to be no significant different between PCA and KPCA. But in common sense, KPCA should be better than PCA. Are there any possible explanations?
dimension dimension? $\endgroup$
dimension reduction? $\endgroup$