0
$\begingroup$

It's my understanding that the weights are updated in a convolutional neural network with each evaluation of a batch.

But when the training data has been processed and it comes to predicting validation set at the end of an epoch, are the weights updated again using the validation error?

For example, is it something like this:

Epoch 1:

Batch 1 > Train > Evaluate > Update Weights
Batch 2 > Train > Evaluate > Update Weights
Batch N > Train > Evaluate > Update Weights

Validation Set > Predict > Evaluate > Update Weights

Epoch 2:

Batch.. etc etc

What I'm wondering is if the weights are also updated at the validation set stage above.

Thanks folks

$\endgroup$
2
$\begingroup$

Validation data is not used to update the weights. If it were used to update the weights, then you would have information leakage because the out-of-sample data is used to fit the model. This defeats the purpose of using a validation set: estimating how well the model does when the model sees new data.

This is true generally, not only in the case of convolutional neural networks (CNNs).

I would describe the process in this way:

For each epoch:

Batch 1 > Forward pass > Backward pass > Update Weights
Batch 2 > Forward pass > Backward pass > Update Weights
...
Batch N > Forward pass > Backward pass > Update Weights

Validation Set > Predict > Evaluate
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.