While I know that some of the classification techniques such as k-nearest neighbour classifier suffer from the curse of high dimensionality, I wonder does the same apply to the support vector machines (SVM)?
I can imagine SVM would only have to take dot products of support vectors and a feature vector to classify a new data point, so it might not suffer from curse of dimensionality. But what about non-linear SVMs?
SVM also suffers the problems coming from high dimensionality, but under typical settings to a lesser degree compared to (say) LDA.
I can imagine SVM would only have to take dot products of support vectors and a feature vector to classify a new data point, so it might not suffer from curse of dimensionality.
This is not the reason for being affected by high dimensionality. High dimensional data affects the identification of support vectors themselves.
Under high dimensional + low sample-size contexts a lot of interesting things start happening. For SVM one such thing is that a lot of data points will become support-vectors at the same time. This most likely will not be the property of the data, hence SVM will be over-fitted.
One way to check for over-fitting in high dimensional data is to see how the points are distributed after the projection. If you find that the points from both classes accumulate right on the margin - then your classification rule is most likely affected.
For more in detail explanation and an example of affected SVM result, see figure 1 from this paper:
J. S Marron, Michael J Todd & Jeongyoun Ahn (2007) Distance-Weighted Discrimination, Journal of the American Statistical Association, 102:480, 1267-1271, DOI: 10.1198/016214507000001120
