Skip to main content
Cross Validated

Return to Answer

edited body
Source Link
Sycorax ♦
Sycorax ♦
  • 94.1k
  • 23
  • 236
  • 390
  1. The matrix dimensions are unrelated to the maximum length of the input. This becomes clear if you examine the equations for whatever RNN variety you're interested in: they all have recurrence in common, so the same weights are re-used to predict $t+1$ using data at $t$ and hidden state information $h_t$.
  2. The hidden weight dimension $N \times N$$H \times H$ refers to the number of units in the hidden layers: $N$$H$ inputs mapped to $N$$H$ outputs.
  1. The matrix dimensions are unrelated to the maximum length of the input. This becomes clear if you examine the equations for whatever RNN variety you're interested in: they all have recurrence in common, so the same weights are re-used to predict $t+1$ using data at $t$ and hidden state information $h_t$.
  2. The hidden weight dimension $N \times N$ refers to the number of units in the hidden layers: $N$ inputs mapped to $N$ outputs.
  1. The matrix dimensions are unrelated to the maximum length of the input. This becomes clear if you examine the equations for whatever RNN variety you're interested in: they all have recurrence in common, so the same weights are re-used to predict $t+1$ using data at $t$ and hidden state information $h_t$.
  2. The hidden weight dimension $H \times H$ refers to the number of units in the hidden layers: $H$ inputs mapped to $H$ outputs.
Source Link
Sycorax ♦
Sycorax ♦
  • 94.1k
  • 23
  • 236
  • 390

  1. The matrix dimensions are unrelated to the maximum length of the input. This becomes clear if you examine the equations for whatever RNN variety you're interested in: they all have recurrence in common, so the same weights are re-used to predict $t+1$ using data at $t$ and hidden state information $h_t$.
  2. The hidden weight dimension $N \times N$ refers to the number of units in the hidden layers: $N$ inputs mapped to $N$ outputs.