Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have a data set that has many categorical values, I want to build a linear model using Logistic regression algorithm.
One way of handling Categorical variables is to use Target Encoder (among others like OneHotEncodng or LabelEncoding).
Is this method suitable for a logistic regression model ? What should we consider while using this type of encoding ?
Surely you can do so, but why? Maybe some ideas here: Strange encoding for categorical features can help, Or if the problem is that you have a categorical variable with very many levels, see Principled way of collapsing categorical variables with many levels?.
One problem with target encoding is that you effectively encodes a variable with its marginal effect. In the case that variable works better together with some other variables, maybe you will not detect that? In addition, if one level has few observations, the encoding will be noisy.
Whenever you're using any encoding that uses the target, you'll want to make sure that you're careful about not using leakage. Especially if you're using cross-fold validation you'll want that encoder to be "trained" only on the portion of the training data for each iteration. This is where you might use sklearn's Pipelines to help.
Specifically to address your question, my favorite technique for pairing with Logistic Regression is Weight-of-Evidence encoding. I typically like to add some random noise as well. If your dataset is small, or your categorical variables have extremely high cardinality, you may not get the best results. However, this is my "favorite" method of encoding that pairs especially well with Logistic Regression.
One of the best packages I've found in python is CategoryEncoders.
