1
$\begingroup$

Let's say I have a Dataset with a coulum called countries. Lots of the values are usa and there is a small amount of values wich are either ger or fra.

Let's also assume train_test_split gives me the following setup

X_train=DataFrame({'country': ['usa']*95+['fra']*5})
X_test=DataFrame({'country': ['usa']*2+['ger']*1+['fra']*3})
y_train=DataFrame({'y': [1]*95+[0]*5})
y_test=DataFrame({'y': [1]*2+[0]*1+[1]*3})

Now I am creating a LabelBinarizer() and fit the X_train on it. Afterwards I run a logistic regression

country_binarizer=LabelBinarizer().fit(X_train)
log_reg=LogisticRegression().fit(country_binarizer.transform(X_train),y_train)

When I now try to predict the accuracy of the Model on the test set i get a ValueError: X has 2 features per sample; expecting 1, since the country_binarizer returns a Matrix with shape (6,2)

 log_reg.score(country_binarizer.transform(X_test), y_test)

How could i tackle this problem. Fitting the LabelBinarizer on the whole Dataset would result in data leakage, or does it?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.