enter image description here

Specifically for Gated Recurrent Unit, and say GRU is "layered" via

enter image description here

but suppose it's only 2 layers deep for simplicity,

and suppose the "total loss" = $L$ = $\sum{l_{t}} = \sum{error(y^{2}_{t})}$ for all $t$,

Then I know how to train the weights for a single layer, so is it possible to train the GRU layer by layer?

In a sense that, find the derivatives wrt the 2nd layer's inputs, thus find $\partial L \over \partial X^{2}_{t}$, for all $t$, which is identical to $\partial L \over \partial h^{1}_{t}$, (thus, the derivatives wrt the 2nd layer's GRU inputs are to be used as the derivatives wrt to the 1st layer's GRU's output) then train the 2nd layer's GRU, then using the derivatives wrt to the 2nd layer's inputs, use the same method that I used to train the 2nd layer, into training the 1st layer's GRU?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.