Questions tagged [gru]
A Gated Recurrent Unit (GRU) is a type of unit in a recurrent neural network.
37
questions
1
vote
0
answers
28
views
Why the number of weights in simple RNN is more than the number of weights in GRU? [closed]
I have read that the number of (trainable) weights/parameters are more in simple RNN compared to GRUs even though GRU also has gates present in it, can anyone explain? Thanks
- 11
0
votes
0
answers
25
views
0
votes
0
answers
12
views
GRU backpropagation [duplicate]
I'm really newbie in deep learning and recently i found GRU method that can use for sentiment analysis. And I confused about GRU backpropagation. can anyone tell me how backpropagation gru formula ...
- 57
0
votes
0
answers
176
views
Backpropagation through time formula on GRU
i searched the literature and found the backpropagation through time formula on LSTM as follows:
in the above formula, input gate, hidden state, output gate, input, cell state have derivative ...
- 57
3
votes
1
answer
241
views
GRU unit: difference between Update and Forget gates
In GRU units,
I don't understand the effective difference between the update and reset gate, $z_t$ and $r_t$ respectively.
\begin{align}
z_t &= \sigma_g(W_{z} x_t + U_{z} h_{t-1} + b_z) \\
r_t &...
- 747
3
votes
1
answer
446
views
To understand the underlying math of GRU neural network in TensorFlow Keras
I tried out a simple GRU network with only 1 layer, 1 input tensor and 1 output, to verify its actual network connection (input nodes->hidden layer->output) by doing the manual calculation with ...
- 31
1
vote
0
answers
83
views
when i have to calculate mape rmse before or after inverse_transform
hello i want to know when i have to calculate mape rmse before this code
predictions1 =scaler.inverse_transform(predictions)or after it. because the result is ...
4
votes
2
answers
2k
views
How GRU solves vanishing gradient
I am learning the GRU model in deep learning and reading this article where details of BPTT are explained. Towards the end the author explained the values of the partial derivative $\frac{\partial h_i}...
- 330
2
votes
1
answer
359
views
How does one initialize an LSTM/GRU to produce the identity function?
I know this sounds silly, but I would like to initialize an LSTM/GRU in such a way that the output hidden state for an arbitrary sequence length is precisely the first $k$ dimensions of the input ...
- 131
2
votes
1
answer
720
views
Number of cells in an RNN
I have been reading about RNNs and I have some confusion between the number of timesteps and number of units in an RNN layer (which after searching for an answer) seems to be a common thing.
I ...
- 31
0
votes
0
answers
52
views
Applying the activation function in this way is equivalent to the other way - RNN
I don't understand why in RNN the 2 following ways of applying the activation functions are equivalent:
First way:
$$
h_t = W\sigma(h_{t-1}) + U x_t + b
$$
Second way:
$$
g_t = \sigma(Wg_{t-1} + U x_t ...
- 227
3
votes
1
answer
478
views
GRU Hidden State Output Formula Difference
Looking into GRU equation I see 2 type for final output. One is from, https://d2l.ai/chapter_recurrent-modern/gru.html#hidden-state, that is,
$
\mathbf{R}_t = \sigma(\mathbf{X}_t \mathbf{W}_{xr} + \...
- 131
4
votes
0
answers
2k
views
RNN vs ResNet for multivariate time series prediction
All others being equal, would a ResNet-based or RNN-based neural network (with/without an attention mechanism) perform better for forecasting a multivariate time series?
Related:
Deep learning for ...
- 267
0
votes
0
answers
51
views
Why in gated recurrent unit gates are controlled by only one layer perceptrons?
Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
- 1
2
votes
1
answer
781
views
Parameters count in GRU layer keras [duplicate]
I have this model structure and want to know the formula for calculating the parameters count in the GRU layer. I did not find that in the docs.
...
- 21
4
votes
1
answer
284
views
Can I use a LSTM Autoencoder to compute similarity between two variable-length audio signals?
I would like to compute the similarity between audio signals of different length.
One way of doing it is to train a RNN (LSTM/GRU) Autoencoder and extract the hidden layer representation - feature ...
0
votes
2
answers
364
views
Is Elmo equivalent to Fasttext+Bi-directional GRU?
From what I have read, Elmo uses bi-directional LSTM layers to give contextual embeddings for words in a sentence. So if I use a bi-directional LSTM/GRU layer over Fasttext representations of words, ...
- 168
1
vote
1
answer
495
views
Given the universal approximation theorem, why are LSTM better than feed forward neural networks at certain tasks?
Per the universal approximation theorem, feed forward neural networks can approximate any function up to an arbitrary level of precision on the domain that they are trained on, given a sufficient ...
- 1,371
0
votes
1
answer
206
views
How to train a RNN language model?
I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction.
How to split the sentences from the dataset into input and ground truth during the ...
- 1
1
vote
1
answer
2k
views
About RNN with variable length output vectors
I have several thousands samples with equal number of features (5000, they are time dependent) and I would like to predict of vectors with variable length.
1) I'm beginner in RNN, and I'd like to ...
- 11
6
votes
1
answer
1k
views
State-of-the-art algorithms for the training of neural networks with GRU or LSTM units
I recently read a lot about neural networks using GRU or LSTM units. There are many easy to use frameworks like tensorflow that do not even require high knowledge about programming. Unfortunately, I ...
- 63
4
votes
1
answer
273
views
What Possible Models For Time-Series when Data is Scarce
Each financial quarter I collect data on the number of potential clients, contacted potential clients, and potential clients that become actual clients. I have this quarterly data going back only 6 ...
- 143
0
votes
2
answers
539
views
Transfer learning for audio
I know that when working with images, what people normally do is download a big model trained with huge data and freeze most of the layers except the lasts ones to train them with their own data.
I'm ...
- 115
1
vote
1
answer
300
views
Huge difference in performances between GRUs and LSTMs. How is it possible?
I'm currently working on a Polyphonic Sound Event Detection task and I've already implemented in Lasagne a complex structure which involves both Convolutional and Recurrent layers. The network is ...
- 26
1
vote
1
answer
177
views
are predictions in a GRU unit based on candidate cell value, or the final memory cell value?
I am learning about GRU's, and something doesn't make sense to me. I've attached the slide that this question is based on (this slide is taken from the "Gated Recurrent Unit (GRU)" video in Andrew Ng'...
- 1,036
9
votes
2
answers
17k
views
What is the output of a tf.nn.dynamic_rnn()?
I am not sure about what I understand from the official documentation, which says:
Returns:
A pair (outputs, state) where:
outputs: The RNN output Tensor.
...
- 2,385
15
votes
2
answers
16k
views
How many parameters are in a gated recurrent unit (GRU) recurrent neural network (RNN) layer?
The title says it all -- how many trainable parameters are there in a GRU layer? This kind of question comes up a lot when attempting to compare models of different RNN layer types, such as long short-...
- 88.9k
1
vote
1
answer
50
views
Has there been any research into using Recurrent Neural Networks (LSTMs, GRUs) with multiple prior time-step inputs/outputs?
RNNs and their variations use the previous time-step output and input, but has there ever been any research done into using multiple time-steps (ie, using each time-step going 10 time steps back) ?
...
- 1,029
3
votes
2
answers
2k
views
Is there any difference between forget gates and remember gates?
In this youtube tutorial by nervada the following diagrams can be seen:
The remember gate in Gated Recurrent Units (GRUs) in the diagram to the right seems to have an analogous function to the forget ...
- 25.7k
4
votes
1
answer
3k
views
How does Keras generate an LSTM layer. What's the dimensionality? [closed]
In Keras I can define the input shape of an LSTM (and GRU) layers by defining the number of training data sets inside my batch (batch_size), the number of time steps and the number of features.
So I ...
- 161
2
votes
1
answer
950
views
Recurrent Neural Networks: How to find the optimal parameters?
I am trying to fit a recurrent neural network for a binary classification problem using the 'keras' library (https://keras.io/layers/recurrent/). Now these networks have many parameters to tune.
LSTM
...
- 4,027
0
votes
1
answer
248
views
How do output-target values align when training a recurrent neural network?
I'm trying to wrap my head around some (crucial) details on how recurrent neural networks work and currently I'm having trouble understanding how the inputs align with the outputs when optimizing an ...
- 173
6
votes
1
answer
6k
views
Is anyone stacking LSTM and GRU cells together and why?
TensorFlow allows you to create MultiRNNCell composed sequentially of multiple simple cells (LSTM and GRU). I usually use same type of cell when creating MultiRNNCell but I was wondering if there ...
- 680
11
votes
1
answer
8k
views
Sudden accuracy drop when training LSTM or GRU in Keras
My recurrent neural network (LSTM, resp. GRU) behaves in a way I cannot explain. The training starts and it trains well (the results look quite good) when suddenly accuracy drops (and loss rapidly ...
- 435
40
votes
5
answers
100k
views
Difference between feedback RNN and LSTM/GRU
I am trying to understand different Recurrent Neural Network (RNN) architectures to be applied to time series data and I am getting a bit confused with the different names that are frequently used ...
- 503
1
vote
3
answers
12k
views
Recurrent Neural Network (LSTM/GRU) in Matlab? [closed]
I wish to explore Gated Recurrent Neural Networks (e.g. LSTM) in Matlab. The closest match I could find for this is the layrecnet. The description for this function is very short and not very clear (i....
- 171
1
vote
1
answer
217
views
What does this gated recurrent unit notation mean?
I'm trying to replicate the algorithm found in this paper:
http://arxiv.org/abs/1506.07285.
And I'm struggling to understand the notation of equation 3 on page 4, which is given as:
Is z(c,m,q) a ...
- 309