7
$\begingroup$

The title says it all -- how many trainable parameters are there in a GRU layer? This kind of question comes up a lot when attempting to compare models of different RNN layer types, such as long short-term memory (LSTM) units vs GRU, in terms of the per-parameter performance. Since a larger number of trainable parameters will generally increase the capacity of the network to learn, comparing alternative models on a per-parameter basis is an apples-to-apples comparison of the relative effectiveness of GRUs and LSTMs.

$\endgroup$
7
$\begingroup$

According to Rahul Dey and Fathi M. Salem, "Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks":

... the total number of parameters in the GRU RNN equals $3 \times (n^2 + nm + n)$.

where $m$ is the input dimension and $n$ is the output dimension. This is due to the fact that there are three sets of operations requiring weight matrices of these sizes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.