Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
My understanding is that regression models are affected by the dummy variable trap. What about other machine learning algorithms e.g. linear svm, logistic regression?
Also, if an algorithm is not affected , is it bad practise to remove a dimension from the dummy variable encoding before training the model?
The dummy variable trap is a redundancy in information in the explanatory variables. Perfect redundancy in explanatory variables is never beneficial. However, some models are hardly affected by it, such as tree-based models, as long as you're solely interested in prediction.
The question then boils down to: How well does my model cope with colinearity?
This has already been asked on this site before for linear SVM. Logistic regression is just a linear model, so there is no reason it would handle colinearity differently than normal linear regression. While you should of course avoid redundancy in dummy variables, even linear regression can still be performed if you use SGD or some form of regularization. The real problem is when you want to use your model for inference, in which case the dummy variable trap can cause small perturbations in the data to lead to large shifts in coefficient estimates.
Fortunately, in software like R you can easily encode factors automatically, without redundancy.
