# Questions tagged [gru]

A Gated Recurrent Unit (GRU) is a type of unit in a recurrent neural network.

37 questions
Filter by
Sorted by
Tagged with
1 vote
28 views

### Why the number of weights in simple RNN is more than the number of weights in GRU? [closed]

I have read that the number of (trainable) weights/parameters are more in simple RNN compared to GRUs even though GRU also has gates present in it, can anyone explain? Thanks
• 11
25 views

...
12 views

### GRU backpropagation [duplicate]

I'm really newbie in deep learning and recently i found GRU method that can use for sentiment analysis. And I confused about GRU backpropagation. can anyone tell me how backpropagation gru formula ...
• 57
176 views

### Backpropagation through time formula on GRU

i searched the literature and found the backpropagation through time formula on LSTM as follows: in the above formula, input gate, hidden state, output gate, input, cell state have derivative ...
• 57
241 views

### GRU unit: difference between Update and Forget gates

In GRU units, I don't understand the effective difference between the update and reset gate, $z_t$ and $r_t$ respectively. \begin{align} z_t &= \sigma_g(W_{z} x_t + U_{z} h_{t-1} + b_z) \\ r_t &...
• 747
446 views

### To understand the underlying math of GRU neural network in TensorFlow Keras

I tried out a simple GRU network with only 1 layer, 1 input tensor and 1 output, to verify its actual network connection (input nodes->hidden layer->output) by doing the manual calculation with ...
• 31
1 vote
83 views

### when i have to calculate mape rmse before or after inverse_transform

hello i want to know when i have to calculate mape rmse before this code predictions1 =scaler.inverse_transform(predictions)or after it. because the result is ...
2k views

• 131
2k views

### RNN vs ResNet for multivariate time series prediction

All others being equal, would a ResNet-based or RNN-based neural network (with/without an attention mechanism) perform better for forecasting a multivariate time series? Related: Deep learning for ...
• 267
51 views

### Why in gated recurrent unit gates are controlled by only one layer perceptrons?

Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
• 1
781 views

### Parameters count in GRU layer keras [duplicate]

I have this model structure and want to know the formula for calculating the parameters count in the GRU layer. I did not find that in the docs. ...
• 21
284 views

### Can I use a LSTM Autoencoder to compute similarity between two variable-length audio signals?

I would like to compute the similarity between audio signals of different length. One way of doing it is to train a RNN (LSTM/GRU) Autoencoder and extract the hidden layer representation - feature ...
364 views

### Is Elmo equivalent to Fasttext+Bi-directional GRU?

From what I have read, Elmo uses bi-directional LSTM layers to give contextual embeddings for words in a sentence. So if I use a bi-directional LSTM/GRU layer over Fasttext representations of words, ...
• 168
1 vote
495 views

### Given the universal approximation theorem, why are LSTM better than feed forward neural networks at certain tasks?

Per the universal approximation theorem, feed forward neural networks can approximate any function up to an arbitrary level of precision on the domain that they are trained on, given a sufficient ...
• 1,371
206 views

### How to train a RNN language model?

I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the ...
1 vote
2k views

### About RNN with variable length output vectors

I have several thousands samples with equal number of features (5000, they are time dependent) and I would like to predict of vectors with variable length. 1) I'm beginner in RNN, and I'd like to ...
1k views

### State-of-the-art algorithms for the training of neural networks with GRU or LSTM units

I recently read a lot about neural networks using GRU or LSTM units. There are many easy to use frameworks like tensorflow that do not even require high knowledge about programming. Unfortunately, I ...
• 63
273 views

### What Possible Models For Time-Series when Data is Scarce

Each financial quarter I collect data on the number of potential clients, contacted potential clients, and potential clients that become actual clients. I have this quarterly data going back only 6 ...
• 143
539 views

### Transfer learning for audio

I know that when working with images, what people normally do is download a big model trained with huge data and freeze most of the layers except the lasts ones to train them with their own data. I'm ...
• 115
1 vote
300 views

### Huge difference in performances between GRUs and LSTMs. How is it possible?

I'm currently working on a Polyphonic Sound Event Detection task and I've already implemented in Lasagne a complex structure which involves both Convolutional and Recurrent layers. The network is ...
1 vote
177 views

### are predictions in a GRU unit based on candidate cell value, or the final memory cell value?

I am learning about GRU's, and something doesn't make sense to me. I've attached the slide that this question is based on (this slide is taken from the "Gated Recurrent Unit (GRU)" video in Andrew Ng'...
• 1,036
17k views

### What is the output of a tf.nn.dynamic_rnn()?

I am not sure about what I understand from the official documentation, which says: Returns: A pair (outputs, state) where: outputs: The RNN output Tensor. ...
• 2,385
16k views

### How many parameters are in a gated recurrent unit (GRU) recurrent neural network (RNN) layer?

The title says it all -- how many trainable parameters are there in a GRU layer? This kind of question comes up a lot when attempting to compare models of different RNN layer types, such as long short-...
• 88.9k
1 vote
50 views

### Has there been any research into using Recurrent Neural Networks (LSTMs, GRUs) with multiple prior time-step inputs/outputs?

RNNs and their variations use the previous time-step output and input, but has there ever been any research done into using multiple time-steps (ie, using each time-step going 10 time steps back) ? ...
• 1,029
2k views

### Is there any difference between forget gates and remember gates?

In this youtube tutorial by nervada the following diagrams can be seen: The remember gate in Gated Recurrent Units (GRUs) in the diagram to the right seems to have an analogous function to the forget ...
• 25.7k
3k views

### How does Keras generate an LSTM layer. What's the dimensionality? [closed]

In Keras I can define the input shape of an LSTM (and GRU) layers by defining the number of training data sets inside my batch (batch_size), the number of time steps and the number of features. So I ...
• 161
950 views

### Recurrent Neural Networks: How to find the optimal parameters?

I am trying to fit a recurrent neural network for a binary classification problem using the 'keras' library (https://keras.io/layers/recurrent/). Now these networks have many parameters to tune. LSTM ...
• 4,027
248 views

### How do output-target values align when training a recurrent neural network?

I'm trying to wrap my head around some (crucial) details on how recurrent neural networks work and currently I'm having trouble understanding how the inputs align with the outputs when optimizing an ...
• 173
6k views

### Is anyone stacking LSTM and GRU cells together and why?

TensorFlow allows you to create MultiRNNCell composed sequentially of multiple simple cells (LSTM and GRU). I usually use same type of cell when creating MultiRNNCell but I was wondering if there ...
• 680
8k views

### Sudden accuracy drop when training LSTM or GRU in Keras

My recurrent neural network (LSTM, resp. GRU) behaves in a way I cannot explain. The training starts and it trains well (the results look quite good) when suddenly accuracy drops (and loss rapidly ...
• 435
100k views

### Difference between feedback RNN and LSTM/GRU

I am trying to understand different Recurrent Neural Network (RNN) architectures to be applied to time series data and I am getting a bit confused with the different names that are frequently used ...
• 503
1 vote
12k views

### Recurrent Neural Network (LSTM/GRU) in Matlab? [closed]

I wish to explore Gated Recurrent Neural Networks (e.g. LSTM) in Matlab. The closest match I could find for this is the layrecnet. The description for this function is very short and not very clear (i....
• 171
1 vote