Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

0
votes
0answers
9 views

Is it possible to make LSTM model with 4dimension shape?

Hellow, wizards. I have time series data including sevaral days. I try to predict a grade of tomorrow, which is range from 0 to 100. And I assume that this grade depends on 3 time-series independent ...
0
votes
0answers
25 views

Number of Hidden Layer Nodes in Recurrent Neural Networks

There's already a decent discussion on how to select the right number of hidden layers and hidden nodes in a feed-forward neural network: How to choose the number of hidden layers and nodes in a ...
0
votes
0answers
14 views

How to train a own question and answer dataset [closed]

I have question and answer data set and i want to train a model so that similar questions will have same answer, How to train the data and which network should i use.. As classification may lead to ...
0
votes
0answers
23 views

Time-series classification of Kinect data using Keras

For my PhD project I recorded using Kinect and Myo 11 people performing Cardiopulmonary Resuscitation (CPR), repeatedly doing chest compressions to a manikin (one person per time). I collected in ...
2
votes
0answers
32 views

Time Series Forecasting RNN: Difference Between Masking & Excluding Rows

Suppose you have missing values in a time series E.g. : t1 x1 y1 t2 ? ? t3 x3 y3 t4 ? ? t5 x5 y5 You are trying to forecast this time series using a recurrent ...
0
votes
0answers
11 views

How to train a RNN language model?

I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the ...
1
vote
1answer
22 views

What is the initial state of the tf.contrib.rnn.LSTMCell? [closed]

Does tf.contrib.rnn.LSTMCell assign itself an initial state of zeros or is it random for each batch or per complete run through (if I run the model twice will it have the same initial state both the ...
0
votes
1answer
15 views

How to understand “multimodal” RNNs for image captioning?

This paper Deep Visual-Semantic Alignments for Generating Image Descriptions on image captioning proposed a Multimodal Recurrent Neural Network architecture. From my understanding, the multimodal RNN ...
0
votes
0answers
16 views

What is the recommended maximum number of time steps for RNN or LSTM?

More time steps incurs longer training time, which above certain limit becomes impractical. What is the recommended maximum limit of the number of time steps for RNN or LSTM? I'm using a powerful ...
0
votes
1answer
19 views

image caption generator

I see two models of image caption generator online: In the above model, the first LSTM cell of decoder takes the entire image as an input. In the above model, all the LSTM cells of the decoder take ...
0
votes
0answers
38 views

LSTMs and Opening/Closing Brackets

I'm training a character-level LSTM to generate molecules using the SMILES system. Each molecule is represented as a string of characters, looking something like this: Cn1c(Nc2c(Cl)ccc(CNC(=O)C(C)(C)...
2
votes
0answers
13 views

How to incorporate uncertainty and noise information in training and prediction of neural networks?

I am trying to use RNNs to perform state estimation on noisy sensor data. The readings are from a GPS dataset and it provides $[longitude, latitude, n_{satellites}]$. The last column, which is the ...
0
votes
0answers
9 views

How does network structure (model complexity) affects covergence speed?

I trained Bi-GRU and HAN (Hierarchical Attention Networks) on my own datasets, and found HAN converges faster than Bi-GRU, within less number of epochs. What would be the reason for this? I guess ...
0
votes
0answers
15 views

how the long term memory works in an LTSM network

I am trying to understand how lstm networks work in particular the long term memory aspect. For example if we have a very long paragraph and at the start we have someone's name, Bob for example. Bob ...
1
vote
1answer
42 views

Selecting number of time lags for input in LSTM networks?

I know from theory that LSTM are meant to selectively capture long and short term dependencies in a sequence. I'm trying to implement LSTMs for a time series task and I notice that a lot of tutorials ...
0
votes
1answer
30 views

About RNN with variable length output vectors

I have several thousands samples with equal number of features (5000, they are time dependent) and I would like to predict of vectors with variable length. 1) I'm beginner in RNN, and I'd like to ...
1
vote
0answers
21 views

Deep Learning Variable Length Sequence Handling

I am trying to understand the best practice for handling different lengths of sequences in NLP tasks. Lets consider an example of convolution on sequences followed by max pool layer. We can handle ...
0
votes
0answers
24 views

Prepare data for stateful/stateless/return_sequences RNN

I'm trying this Tensorflow example that using GRU to predict and generate text. Suppose there is the text "Hello World yes", and the sequence length is 5. Then we prepare the training data to be (...
2
votes
1answer
139 views

How to deal with unknown classes with a neural network classifier?

I have a small RNN with a softmax output, which succesfully classifies sequences within a known set of n classes. Now I have the problem that there might be sequences in the test data which do not ...
0
votes
0answers
13 views

What is the best model for keyphrase extraction from super long text?

I’m working on a keyphrase extraction task. The biggest difficulty of this task is that the text is very long (5000-20000 words). I’ve tried several unsupervised algorithms such as Tf-idf and TextRank ...
2
votes
0answers
62 views

LSTM : multi-step multidimensional multivariate multi-site timeseries forecasting [closed]

I'm working on a project in which i'm trying to do a pollution forecasting. I googled around and found that LSTM is a good candidate for this task, however, I'm still struggling at how to adapt it to ...
2
votes
1answer
56 views

Forecasting algorithms for incomplete time series data [duplicate]

I want to forecast the demand of each SKU in my warehouse every week from the history transaction that I have collected. The data contains brand, product type, SKU, quantity, date(per day), price. But ...
0
votes
0answers
16 views

How to obtain embedded representation of single test instance after training

The first layer of my RNN is embedded layer as follows. ...
0
votes
0answers
28 views

State prediction of how long someone sleeps using neural nets

I have over hundred thousands of datapoints on how long individual people sleep. I also have information about how soft their beds are, their income, stress levels etc. At first I want to predicted ...
0
votes
1answer
22 views

How are RNNs with inputs greater than the defined sequence length implamented

To clarify the slightly ambiguous language in the title. I have an RNN (actually 2 stacked RNN layers) that take input X of size X [batch_size, sequence_length, features] the model is trying to ...
0
votes
1answer
78 views

LSTM output dimensionality

I am new to LSTMs. When reading the papers and websites about LSTM architecture, there is something I do not get. As I understand it, a single LSTM layer can have multiple LSTM cells (just like a ...
0
votes
1answer
69 views

Is this a sign of (bad) local minima encountered by this RNN?

On a binary classification task, I can get perfect performance on the training set, but no matter how strongly I regularize the recurrent neural network (dropout 0.99, L2 weight penalties of 0.01) the ...
1
vote
0answers
37 views

How to apply dropout in LSTMs?

Dropout in fully connected neural networks is simpl to visualize, by just 'dropping' connections between units with some probability set by hyperparamter p. However, how dropout works in recurrent ...
0
votes
0answers
28 views

Use of variable 'states' in a recurrent neural network in tensorflow? [duplicate]

The output of RNN implementations in Tensorflow is a tuple [outputs, states]. I am a bit confused about the use of the states variable. For example in this example of a simple LSTM implementation in ...
0
votes
0answers
17 views

Priming GRU in RNN: Are they doing it in the wrong way?

I am looking at some code (in PyTorch but the question is general) where they use a technique called "priming" in order to "start" the prediction of an RNN that mainly just consists of a single GRU (...
0
votes
1answer
497 views

Time steps in Keras LSTM

My understanding of time-series LSTM training is that the recurrent cell gets unrolled to a specified length (num_steps), and parameter updates are back-propagated ...
0
votes
0answers
34 views

Can attention be implemented without encoder / decoder?

I just got into models beyond biLSTM, would like to start with applying attention to my existing network (RNN). I find examples for attention always with encoder decoder architecture, however is it ...
0
votes
0answers
13 views

Demand forecasting for sequence data and multi-task learning

I have a demand forecasting problem that I'd like to solve with a deep learning using multi-task learning and I'd like advice in some areas. Problem definition: I have a set of $N$ customers that ...
0
votes
0answers
8 views

Bring improvements inside a neural network configuration

Currently, I found the right recipe for a time series regression problem to finally get acceptable to good results. Here is the config file ...
0
votes
1answer
245 views

How to deal with really sparse time series data for a binary classification task using RNN or LSTM?

I have a binary classification prediction task and more often than not, the time series data is like really sparse. The number of zeroes in the time series data is almost always more than 99%. I ...
1
vote
0answers
45 views

A mistake in Tensorflow's documation?

Tensorflow's documentation gives an example for text generation using a RNN with eager execution. To the best of my understanding, this examples defines a simple RNN (with a GRU cell and a projection ...
1
vote
0answers
46 views

How does the internal state of Tensorflow's “dtf.nn.dynamic_rnn()” change?

I am trying to do a uni-variate forecasting and I have come across this question quite a lot. This question is regarding TensorFlow's "dtf.nn.dynamic_rnn()". NOTE: When I say internal state I am ...
0
votes
0answers
6 views

Using character level model to notes generation

I would like to use character based model for music generation. Predict note after N notes. (like character model based text generators). Do to this I would like to use RNN with LSTM in Keras. I ...
3
votes
1answer
100 views

How to minimize sharpe ratio with LSTM recurrent neural network?

I've read some articles about trading using recurrent reinforcement learning such as this one. The point where I do not fully understand is how to construct the cost/loss function. In the article, ...
0
votes
0answers
31 views

Can someone explain RNN weight update derivation?

I am going through this tutorial and not able to understand derivation for weight update. Specifically I am struggling to understand the derivative of vector w.r.t matrix, as according to my knowledge ...
1
vote
0answers
73 views

Variable importance (?) for multivariate time series anomaly detection methods

I'm working on anomaly detection methods for multivariate time series $[\mathbf{x}^{new}_1,\dots,\mathbf{x}^{new}_T]$ where $\mathbf{x}^{new}_{i}$ is $p-$dimensional. I won't go into the details of ...
2
votes
1answer
40 views

Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID

From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
2answers
43 views

In Recurrent NN, what's the reason for adding instead of multiplying the input term and the state term in the hidden units?

As we know, the hidden layer unit has the following activation: $$h_t=tanh(UX_t+Wh_{t-1})$$ So there is the interaction between the input and the previous state: $UX_t+Wh_{t-1}$. My question is why it ...
1
vote
2answers
108 views

Mathematical structure of SimpleRNN in keras

Two types of RNN can be used: Type1: The output is being used as state h(t) = g(W1.x(t) + W2.h(t-1) + b1) Type2: There is a state in addition to the output a(t) = g(W1.x(t) + W2.a(t-1) + b1) h(t) ...
0
votes
1answer
43 views

Testing an LSTM making predictions 1 timestep into the future

Say I have a time series data set of 100 sequential timesteps, and I want to train and test an LSTM on the data set, but only on forecasting a single timestep into the future. I want more than one ...
1
vote
1answer
126 views

Can Neural Network take multidimensional inputs?

I recently learned RNN and find that a common feature of it and CNN is that they use either a LSTM cell or Convolution to process a single multidimensional input (like images and word embeddings which ...
1
vote
0answers
45 views

Interpreting accuracy graph for a LSTM model | Keyword Prediction

I have created a LSTM model for keyword prediction. I am using RMSE optimizer for training. I observe that the train and test accuracy decreases at first and then fluctuates without much difference in ...
0
votes
0answers
83 views

NaN output from Bidirectional LSTM in Keras

I am creating a bi-directional LSTM using tf.keras APIs: input_layer = tf.reshape(emb_seqs, [const.TRN_BATCH_SIZE, -1, const.VECTOR_SIZE]) lstm_layer = tf.keras.layers.LSTM(units=LSTM_UNITS, ...
1
vote
0answers
44 views

How does LSTM learn [duplicate]

I am confused on how LSTM learn from word embedding. I know that LSTM accepts 3D input (sample, timesteps, features). So, when we use embedding layer (word2vec) and we have 300-d vector representation,...
0
votes
1answer
122 views

What kind of a generative model is an RNN?

Given the taxonomy of generative models as presented by Ian Goodfellow in Tutorial on Generative Adversarial Networks (https://arxiv.org/abs/1701.00160), in what branch do we put the family of RNNs?