# Can you train deep recurrent neural network layer by layer?

Specifically for Gated Recurrent Unit, and say GRU is "layered" via

but suppose it's only 2 layers deep for simplicity,

and suppose the "total loss" = $$L$$ = $$\sum{l_{t}} = \sum{error(y^{2}_{t})}$$ for all $$t$$,

Then I know how to train the weights for a single layer, so is it possible to train the GRU layer by layer?

In a sense that, find the derivatives wrt the 2nd layer's inputs, thus find $$\partial L \over \partial X^{2}_{t}$$, for all $$t$$, which is identical to $$\partial L \over \partial h^{1}_{t}$$, (thus, the derivatives wrt the 2nd layer's GRU inputs are to be used as the derivatives wrt to the 1st layer's GRU's output) then train the 2nd layer's GRU, then using the derivatives wrt to the 2nd layer's inputs, use the same method that I used to train the 2nd layer, into training the 1st layer's GRU?