Questions tagged [gru]

A Gated Recurrent Unit (GRU) is a type of unit in a recurrent neural network.

0
votes
0answers
20 views

Recurrent Neural Network - Time Series prediction: How to decide on the sequence length and number of output neurons

Context I am currently working with Gated Recurrent Units for time series prediction. Anyhow, my question should apply to all kinds of such recurrent networks. In my data, I have timestamps of 15 ...
0
votes
1answer
35 views

Given the universal approximation theorem, why are LSTM better than feed forward neural networks at certain tasks?

Per the universal approximation theorem, feed forward neural networks can approximate any function up to an arbitrary level of precision on the domain that they are trained on, given a sufficient ...
0
votes
0answers
22 views

NER with GRUs neural network with imbalance dataset

this is my first time asking question on CrossValidated, so if there is any mistake on my part, i apologize. I will try not to make those mistakes again. I'm trying to do a NER task. The problem is ...
0
votes
0answers
450 views

Number of Hidden Layer Nodes in Recurrent Neural Networks

There's already a decent discussion on how to select the right number of hidden layers and hidden nodes in a feed-forward neural network: How to choose the number of hidden layers and nodes in a ...
0
votes
1answer
38 views

How to train a RNN language model?

I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the ...
0
votes
1answer
268 views

About RNN with variable length output vectors

I have several thousands samples with equal number of features (5000, they are time dependent) and I would like to predict of vectors with variable length. 1) I'm beginner in RNN, and I'd like to ...
0
votes
0answers
37 views

Priming GRU in RNN: Are they doing it in the wrong way?

I am looking at some code (in PyTorch but the question is general) where they use a technique called "priming" in order to "start" the prediction of an RNN that mainly just consists of a single GRU (...
0
votes
0answers
53 views

How to overfit an LSTM for acoustic features?

Aim I would like to train an LSTM to learn the mapping between the EMA signals and audio signals in the MOCHA-TIMIT dataset. I've looked at publications using similar approaches to see if I can ...
4
votes
1answer
386 views

State-of-the-art algorithms for the training of neural networks with GRU or LSTM units

I recently read a lot about neural networks using GRU or LSTM units. There are many easy to use frameworks like tensorflow that do not even require high knowledge about programming. Unfortunately, I ...
4
votes
1answer
76 views

What Possible Models For Time-Series when Data is Scarce

Each financial quarter I collect data on the number of potential clients, contacted potential clients, and potential clients that become actual clients. I have this quarterly data going back only 6 ...
0
votes
1answer
280 views

Transfer learning for audio

I know that when working with images, what people normally do is download a big model trained with huge data and freeze most of the layers except the lasts ones to train them with their own data. I'm ...
1
vote
1answer
148 views

Huge difference in performances between GRUs and LSTMs. How is it possible?

I'm currently working on a Polyphonic Sound Event Detection task and I've already implemented in Lasagne a complex structure which involves both Convolutional and Recurrent layers. The network is ...
1
vote
1answer
78 views

are predictions in a GRU unit based on candidate cell value, or the final memory cell value?

I am learning about GRU's, and something doesn't make sense to me. I've attached the slide that this question is based on (this slide is taken from the "Gated Recurrent Unit (GRU)" video in Andrew Ng'...
8
votes
2answers
10k views

What is the output of a tf.nn.dynamic_rnn()?

I am not sure about what I understand from the official documentation, which says: Returns: A pair (outputs, state) where: outputs: The RNN output Tensor. ...
7
votes
1answer
3k views

How many parameters are in a gated recurrent unit (GRU) recurrent neural network (RNN) layer?

The title says it all -- how many trainable parameters are there in a GRU layer? This kind of question comes up a lot when attempting to compare models of different RNN layer types, such as long short-...
1
vote
0answers
704 views

From One Layer GRU to Multi-Layer GRU with tensorflow [closed]

I want to improve my prediction with an RNN-GRU. Right now I have a 1-layer GRU with 2 outputs: ...
0
votes
1answer
36 views

Has there been any research into using Recurrent Neural Networks (LSTMs, GRUs) with multiple prior time-step inputs/outputs?

RNNs and their variations use the previous time-step output and input, but has there ever been any research done into using multiple time-steps (ie, using each time-step going 10 time steps back) ? ...
2
votes
1answer
886 views

Is there any difference between forget gates and remember gates?

In this youtube tutorial by nervada the following diagrams can be seen: The remember gate in Gated Recurrent Units (GRUs) in the diagram to the right seems to have an analogous function to the forget ...
4
votes
1answer
2k views

How does Keras generate an LSTM layer. What's the dimensionality? [closed]

In Keras I can define the input shape of an LSTM (and GRU) layers by defining the number of training data sets inside my batch (batch_size), the number of time steps and the number of features. So I ...
1
vote
1answer
494 views

Recurrent Neural Networks: How to find the optimal parameters?

I am trying to fit a recurrent neural network for a binary classification problem using the 'keras' library (https://keras.io/layers/recurrent/). Now these networks have many parameters to tune. LSTM ...
0
votes
1answer
134 views

How do output-target values align when training a recurrent neural network?

I'm trying to wrap my head around some (crucial) details on how recurrent neural networks work and currently I'm having trouble understanding how the inputs align with the outputs when optimizing an ...
4
votes
1answer
2k views

Is anyone stacking LSTM and GRU cells together and why?

TensorFlow allows you to create MultiRNNCell composed sequentially of multiple simple cells (LSTM and GRU). I usually use same type of cell when creating MultiRNNCell but I was wondering if there ...
8
votes
1answer
2k views

Sudden accuracy drop when training LSTM or GRU in Keras

My recurrent neural network (LSTM, resp. GRU) behaves in a way I cannot explain. The training starts and it trains well (the results look quite good) when suddenly accuracy drops (and loss rapidly ...
1
vote
3answers
9k views

Recurrent Neural Network (LSTM/GRU) in Matlab? [closed]

I wish to explore Gated Recurrent Neural Networks (e.g. LSTM) in Matlab. The closest match I could find for this is the layrecnet. The description for this function is very short and not very clear (i....
1
vote
1answer
172 views

What does this gated recurrent unit notation mean?

I'm trying to replicate the algorithm found in this paper: http://arxiv.org/abs/1506.07285. And I'm struggling to understand the notation of equation 3 on page 4, which is given as: Is z(c,m,q) a ...