I'm learning about SVM's and understand that boosting something into a higher dimension can sometimes help separate the data better. However, if I were to perform 1 nearest neighbor with the RBF kernel, is it possible that the classification performance is better in the higher dimensional space than the lower dimensional space?
In general RRF kernels perform better but they are prone to over-fitting. In addition you must also perform free parameter selection. In the paper Kernel Nearest-Neighbor Algorithm that is to my knowledge the first paper on kernel -KNN the following is stated:
"If samples distribute arbitrarily, conventional nearest-neighbor algorithm may not obtain satisfactory result. However, mapping to a high dimensional space, the kernel nearest-neighbor algorithm can work better and obtain good results."
The paper used polynomial kernels due to free parameter issues but mapping to a high dimensional space was helpful.