1
$\begingroup$

While doing hyperparameter tuning to find appropriate k, we use validation error to find k. Then what is the use of training error here?

$\endgroup$

2 Answers 2

0
$\begingroup$

When doing cross-validation you care about validation error, not the train error. With $k$-NN, you would get perfect train error with $k=1$ i.e. returning the same sample as your prediction. Train error may be useful to judge overfitting, when the difference between train and validation error is relatively big.

$\endgroup$
0
$\begingroup$

The training error is what is being minimized internally with the optimization algorithm, so it is not useless. The optimization algorithm in itself is unaware of the validation error. The model selection algorithm is what uses the validation error. Together (optimization + model selection) they make up the learning algorithm.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.