Is Support Vector Machine sensitive to the correlation between the attributes? I would like to train an SVM to classify cases (TRUE/FALSE) based on 20 attributes. I know that some of those attributes are highly correlated. Therefore my question is: is SVM sensitive to the correlation, or redundancy, between the features? Any reference?
 A: Linear kernel: The effect here is similar to that of multicollinearity in linear regression. Your learned model may not be particularly stable against small variations in the training set, because different weight vectors will have similar outputs. The training set predictions, though, will be fairly stable, and so will test predictions if they come from the same distribution.
RBF kernel: The RBF kernel only looks at distances between data points. Thus, imagine you actually have 11 attributes, but one of them is repeated 10 times (a pretty extreme case). Then that repeated attribute will contribute 10 times as much to the distance as any other attribute, and the learned model will probably be much more impacted by that feature.
One simple way to discount correlations with an RBF kernel is to use the Mahalanobis distance: $d(x, y) = \sqrt{ (x - y)^T S^{-1} (x - y) }$, where $S$ is an estimate of the sample covariance matrix. Equivalently, map all your vectors $x$ to $C x$ and then use the regular RBF kernel, where $C$ is such that $S^{-1} = C^T C$, e.g. the Cholesky decomposition of $S^{-1}$.
