0
$\begingroup$

I've coded a logistic regression classifier trained on the Kaggle Titanic dataset to predict whether or not a passenger will survive.

I've noticed that depending on how I represent the gender the accuracy of my classifier changes. So Males as 1 and Females 0, rather than vice versa, gives me a 2% bump in accuracy, While If I represent M:1 and Females:3, the accuracy jumps by 5-6%

I was wondering why this might be the case? and what are the best practices when it comes to representing data like this?

$\endgroup$

1 Answer 1

1
$\begingroup$

The best practice is to convert to a column that's 1 if it's a male, 0 if it's a female. You may want to switch these, I think 1 for male is more common.

Other encodings are used, for example -1 and 1. This influences the interpretation of the coefficients that are fitted with a regression model.

Other than the interpretation and the actual value of the coefficients, the model itself should not change when you use other encodings, at least not in a logistic regression model. But there are many ways in which that might not hold exactly, for example, numeric unstability, or inadvertent use of some regularization constant, or application of scaling.

Numeric unstability is very unlikely, but you can see that if you use very small or very large numbers, you might run into problems if the implementation you use doesn't cope with it.

If you use regularisation, then different encodings can really make a difference. This happens if you use LASSO or elastic net or something like that. This is likely, because many implementations use some regularization by default, for example in sklearn: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html.

You can also apply scaling to the training set and the test set in many ways, some more correct than others. If you have a training set with mostly female, then apply scaling, it will influence the effective encoding, to something like 5-male and 0-female. In combination with the above, this may make a difference. In addition, you may mix up test- and trainingset with the scaling in various ways, and further influence the outcomes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.