When creating a neural network, do I update the weights after each run of forward then back propogation? Or do I just keep the random weights and update the Delta variables?
I am looking at slide 8 on these notes:https://d396qusza40orc.cloudfront.net/ml/docs/slides/Lecture9.pdf
It says:
For i = m
Set a(1) = x(i)
Perform Forward-Propogation
Compute delta
Compute DELTA
QUESTION: Do I update the Weights that I use in Forward-propogation, or do I
use random weights and just keep updating the accumulator 'DELTA'? And if I
update the weights, do I set them to DELTA?