In a multi layer perceptron or feedforward neural network, isn't backpropagation updating weights of the middle layers that are dependent variables? So for a particular hidden layer, it calculates all of the partial derivatives of all of the nodes and updates the weights by a learning rate/optimizer. But isn't it updating the partial derivatives of functions that are dependent variables? So if all the input values to that layer are fixed, then updating all of the partial derivatives of the hidden node is correct because that is the best thing to do to decrease the loss. But we are updating the previous layer(s) as well, so the input values for the particular hidden node after the previous layer(s) are updated, are no longer the same values so it updated the weights of the particular hidden node on input values that are no longer used.
Do anyone have insight on this? Maybe I'm totally off?
Thanks so much for the help.