Let's say I have a Dataset with a coulum called countries. Lots of the values are usa and there is a small amount of values wich are either ger or fra.

Let's also assume train_test_split gives me the following setup

X_train=DataFrame({'country': ['usa']*95+['fra']*5})
X_test=DataFrame({'country': ['usa']*2+['ger']*1+['fra']*3})
y_train=DataFrame({'y': [1]*95+[0]*5})
y_test=DataFrame({'y': [1]*2+[0]*1+[1]*3})

Now I am creating a LabelBinarizer() and fit the X_train on it. Afterwards I run a logistic regression


When I now try to predict the accuracy of the Model on the test set i get a ValueError: X has 2 features per sample; expecting 1, since the country_binarizer returns a Matrix with shape (6,2)

 log_reg.score(country_binarizer.transform(X_test), y_test)

How could i tackle this problem. Fitting the LabelBinarizer on the whole Dataset would result in data leakage, or does it?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.