1
$\begingroup$

I am trying to figure out how to build my model's vocabulary in the case of K-Fold Cross validation. In general I believe, the vocabulary should be built based on training data only. However in the K-Fold Cross validation setting, the training set changes repeadetly and thus the model have a different vocabulary if I ignore the validation set while building vocabulary.

What I wonder is that, even though it cause such a thing should I still ignore the validation set while building the vocabulary or can vocabulary be built by using combination of training and validation sets ?

$\endgroup$

1 Answer 1

1
$\begingroup$

Keep in mind what the purpose of the validation process is: You simulate what happens if the model has to make predictions on data it may have not seen before, thus estimating its generalization power.

If you build the vocabulary on the whole data set you may overestimate the generalization power of the model. So this is not an option. If the model fluctuates too much you either need more data or find a model which is less sensitive to data variations or try out Leave-One-Out (the most extreme form of cross-validation, but computationally expensive).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.