1
$\begingroup$

In my sentiment analysis work ı have used k fold cross validation, and ı got below results

precision recall f1-score support

       0      0.906     0.916     0.911       878
       1      0.865     0.845     0.855      1142
       2      0.849     0.864     0.857       950

accuracy                          0.872      2970

macro avg 0.873 0.875 0.874 2970 weighted avg 0.872 0.872 0.872 2970

val-accuracy for each fold [0.7805452709525412, 0.8114478114478114, 0.8454545454545455, 0.8494949494949495, 0.8720538720538721]

average val-accuracy 0.8317992898807439

I wonder that what is my model accuracy, ı got 87 for last fold but ı got only about 70s when ı tested my model with unseen data. I think when we test with unseen ı am getting results like first fold. For kfold cross validation after first fold the model knows every data, because of that reason at second or third... fold we got more accuracy,am ı true? ıf ı am true kfold doesnt increse accuracy

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Kfold is not used for increasing accuracy, it is used to shuffle your data and then test your estimator, your predefined parameters in the model. It gives you an insight how your model behaves. If you have vastly changes in accuracy/scores in a kfold, you may have a look at outliers in your data as this or these outliers may sometimes jump from one fold into the other.

In summary:Kfold

Kfold takes different subsamples of your data to use as test_data, thus there is no exact acuracy score.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answers, patrick. I want to ask another thing, When ı investigate some questions about kfold some contributors says that each fold is used to train a new model from scratch, predict the accuracy, and then the model is discarded. But when ı used k fold cv, ı see that accuracy increasing each fold, smallest one is initial fold. is it just a coincidicence? $\endgroup$
    – Zafer Demir
    Commented Mar 13, 2021 at 12:11
  • $\begingroup$ I do not know your data, but it sounds you didn't shuffled your data, which could be an indicator to your phenomenon as the relationship grows towards the end of the data, thus the latest split, try to insert a shuffle statology.org/k-fold-cross-validation-in-python $\endgroup$
    – Patrick Bormann
    Commented Mar 13, 2021 at 12:43
  • $\begingroup$ Acctually ı shuffled data at kfold definition, also ı shuffled data when loading my csv data, ı still have above problem . ı saw some recommendations that advise using built in cross_val_score. In that case ı dont know how to use my model attributes like epoch , batch szie, call backs, because cross_val_score doesnt have them. $\endgroup$
    – Zafer Demir
    Commented Mar 13, 2021 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.