It seems that a single-layer radial basis function network with normalized weights is the same thing as kernel smoothing (see e.g. Haykin Neural Networks: a Comprehensive Foundation, Section 5.12).
It's well known that local smoothing methods don't work in high dimensions (say, more than ten dimensions) due to the curse of dimensionality. For example, $k$-nearest neighbors doesn't work very well at all (see e.g. Hastie, Tibshirani, and Friedman, Elements of Statistical Learning, section 2.5).
I have two questions:
I presume Nadaraya-Watson kernel smoothing suffers from the same problem as kNN. Is this the case? And the same for radial basis function networks: can they be used in high dimensions?
If these methods can't be used in high dimensions, then why do "kernelized" regression methods (like SVM or kernel ridge regression) work in high dimensions? In practice, people use them successfully on high dimensional inputs like images?