Let's say we have a standard classification problem where we want to classify samples into two groups based on some number of predictors.
Is it possible to do this with above-chance accuracy, if there is absolutely no difference between the two target groups in means of each individual predictors using linear SVM?
My reasoning is, that means is quite sensitive to outliers or just long tails and differences in distributions, but hinge loss of SVM should be more robust. On the other hand, SVM cares about the margin, which should be different than just looking at means.
I was trying to classify some distributions with the same means, but outliers, however, I could never produce an example where linear SVM would be able to do this. Can such an example be created? If not, why not?
Edit: Here is an example of a situation where means of orange and blue cases for both x1 and x2 variables are exactly 0 (due to blue outliers), although it is easy to imagine quite a good decision boundary. However, SVM seems not to be able to reliably classify sample in this example.