2
$\begingroup$

My understanding is that regression models are affected by the dummy variable trap. What about other machine learning algorithms e.g. linear svm, logistic regression?

Also, if an algorithm is not affected , is it bad practise to remove a dimension from the dummy variable encoding before training the model?

$\endgroup$
5
$\begingroup$

The dummy variable trap is a redundancy in information in the explanatory variables. Perfect redundancy in explanatory variables is never beneficial. However, some models are hardly affected by it, such as tree-based models, as long as you're solely interested in prediction.

The question then boils down to: How well does my model cope with colinearity?

This has already been asked on this site before for linear SVM. Logistic regression is just a linear model, so there is no reason it would handle colinearity differently than normal linear regression. While you should of course avoid redundancy in dummy variables, even linear regression can still be performed if you use SGD or some form of regularization. The real problem is when you want to use your model for inference, in which case the dummy variable trap can cause small perturbations in the data to lead to large shifts in coefficient estimates.

Fortunately, in software like R you can easily encode factors automatically, without redundancy.

$\endgroup$
1
  • 1
    $\begingroup$ thanks a lot for your answer. From the link that you provided I understand that both linear and non linear svm is affected by the dummy variable trap. $\endgroup$ – Stamatis Tiniakos Jul 3 '19 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.