I'm training a support-vector machine (SVM). Each training vector includes 2 features which are equal in magnitude and have opposite signs, i.e., $F_1 = -F_2$.
Is there any sense in doing so?
Is one of the features redundant and better to be removed?
Are the answers to the above dependent on the kernel being used? I'm currently using the Gaussian/radial-basis function (RBF) kernel.
Consider an RBF kernel with different scale factors for each feature
$\mathcal{K}(\vec{x},\vec{x}') = \exp\left\{\sum_{i=1}^d \eta_i(x_i - x_i')^2\right\}$
Then duplicting a feature with the a different sign is equivalent to just doubling the scale factor for that feature. So if you had perfect model selection (which chooses optimal values for all $\eta_i$ and C), then adding the extra feature would have no effect. However, model selection for SVMs is more tricky than it looks as while the SVM is based on theory that give protection against over-fitting in fitting the model, it provides not protection against over-fitting in model selection, so in practice it probably will make a difference.
If the dataset is large and you have many features it probably won't make much difference.
I would leave the duplicate out as it provides no additional information.
