1
$\begingroup$

I've been working on a simple logistic regression model and I'm trying to improve its precision by cross validation. This is the code I've done so far (without including the imports):

csv_data=pd.read_csv(path)
df=pd.DataFrame(csv_data)
X = df.drop(["label"],axis=1)
y = df["label"]
logmodel=LogisticRegression()

X_train, X_test, y_train, y_test = 
train_test_split(X,y,test_size=0.2,random_state=0)
logmodel.fit(X_train,y_train)
predictions = logmodel.predict(X_test)

grid_param={
    "penalty" : ["l1","l2"],
    "C"       : np.logspace(-1, 1, 100),
}

gd_sr = GridSearchCV(estimator=logmodel,  
                     param_grid=grid_param,
                     scoring='accuracy',
                     cv=5,
                     n_jobs=-1)

gd_sr.fit(X_train, y_train) 

best_parameters = gd_sr.best_params_  
best_result = gd_sr.best_score_  
print("Before Cross Validation: " +str(accuracy_score(y_test,predictions)))
print("After Cross Validation: " + str(best_result))

When executing this code the output is:

Before Cross Validation: 0.8082191780821918
After Cross Validation: 0.7821612349914236

As you can see, the precission drops after using cross validation. Is this normal? My intuition tells me that after using cross validation, the model would always score better or equal than without using it. Am I doing something wrong?

$\endgroup$
  • 3
    $\begingroup$ Be sure to understand what you're doing: CV does not "improve precision". It's a re-sampling method and, after checking your code, you are using it to perform a grid-search to estimate the best model parameters. Also, accuracy and precision are two very different things! $\endgroup$ – bi_scholar May 24 at 14:25
  • $\begingroup$ I understood that when using grid-search, you find the model parameters that will make your model be more precise. Is that false? Does it improve accuracy instead? $\endgroup$ – Jaume Brossa Ordóñez May 24 at 15:13
  • $\begingroup$ I just used 'logmodel.score(X_test, y_test)' to compare accuracy and it improved after using cross validation. I understand now the difference, thank you @bi_scholar! $\endgroup$ – Jaume Brossa Ordóñez May 24 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.