I want to implement an algorithm in a paper which uses kernel SVD to decompose a data matrix. So I have been reading materials about kernel methods and kernel PCA etc. But it still is very obscure to me especially when it comes to mathematical details, and I have a few questions.
Why kernel methods? Or, what are the benefits of kernel methods? What is the intuitive purpose?
Is it assuming a much higher dimensional space is more realistic in real world problems and able reveal the nonlinear relations in the data, compared to non-kernel methods? According to the materials, kernel methods project the data onto a high-dimensional feature space, but they need not to compute the new feature space explicitly. Instead, it is enough to compute only the inner products between the images of all pairs of data points in the feature space. So why projecting onto a higher dimensional space?
On the contrary, SVD reduces the feature space. Why do they do it in different directions? Kernel methods seek higher dimension, while SVD seeks lower dimension. To me it sounds weird to combine them. According to the paper I am reading (Symeonidis et al. 2010), introducing Kernel SVD instead of SVD can address the sparsity problem in the data, improving results.
From the comparison in the figure we can see that KPCA gets an eigenvector with higher variance (eigenvalue) than PCA, I suppose? Because for the the largest difference of the projections of the points onto the eigenvector (new coordinates), KPCA is a circle and PCA is a straight line, so KPCA gets higher variance than PCA. So does it mean KPCA gets higher principal components than PCA?