I understand the math in kernel PCA and with RBF kernel, and I also understand that the RBF kernel map the data into a infinite dimensional space. I know that for SVM, mapping the data into a higher dimensional space will always make it easier to linearly separate the data from different classes.
But what I don't understand is why mapping data into a higher dimensional space (say infinite using RBF kernel) will be more suitable to use linear PCA in the feature space? Is the performance always guaranteed to improve as compare to PCA when applied to nonlinear data?
I tried to visualize the process but it is hard for me when I use the RBF kernel. I don't physically or intuitively think mapping the data into higher-dimensional space will always work for PCA purpose (for nonlinear data) . Is there any concrete example of explanation about what actually (not just illustrative plot) happens and why it helps to use RBF kernel? And is there any way to characterize how data distributed in the feature space by using RBF kernel?
Notes: I know that the theory and applications are well-established for classification purpose, but I don't know for KPCA as a dimensionality reduction technique (or distribution modeling) purpose, and that's the main aim of this question.