After watching 3Blue1Brown's tutorial series, and an array of others, I'm attempting to make my own neural network from scratch.
So far, I'm able to calculate the gradient for each of the weights and biases.
Now that I have the gradient, how am I supposed to correct my weight/bias?
- Add the gradient and the original value?
- Multiply the gradient and the original value?
- Something else? (Most likely answer)
In addition to this, I've been hearing the term learning rate being tossed around, and how it is used to define the magnitude of the 'step' to descend to minimum cost. I figured this may also play a major role in reducing the cost.