Changing the weights values in backprop, by how much?

Question

So im reading through this link on Neural Nets and regression and on this page the part about back propagation which reads:

To perform backpropagation and make the network learn, you simply compare ŷ to the ground-truth value of y and adjust the weights and biases of the network until error is minimized, much as you would with a classifier. Root-means-squared-error (RMSE) could be the loss function.

My question is how much do i adjust the weights and biases by?
And in what order?

Now I know the second part of my question obvious answer is 'backwards' but what i mean is
if im changing the weights from back to front. how can i know that if i change a weight near the input so it wont drastically mess up the Y? So I would have to go back to the weight near the output and change that so the weight near the input dosent make such a drastic change.

Dose this make sense? if not I will clarify.

Answer 1

This paragraph gives a very high-level overview of backpropagation. Backpropagation is a mathematical technique that relates the change of a weight to a change in the outcome.

A good discussion is this chapter. As you can see, a whole chapter has been devoted to explaining it.

At the heart of backpropagation is an expression for the partial derivative $\partial C / \partial w $ of the cost function $C$ with respect to any weight $w$ (or bias $b$) in the network. The expression tells us how quickly the cost changes when we change the weights and biases. And while the expression is somewhat complex, it also has a beauty to it, with each element having a natural, intuitive interpretation.

So the answer is that you adjust all weights and biases simultaneously by the amount calculated using backpropagation.