Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

0
votes
0answers
4 views

Using RNNs to segment long streams

I'm using an LSTM to segment a long streams of linear data into contiguous parts. My specific application is extracting key signature changes from a stream of notes. Specifically, I want to (for each ...
0
votes
0answers
11 views

How can fix outliers prediction result from RNN model output in Keras?

I'm dealing with RNN models by normalized data in range of [-1,+1] and reshaped data for each time sequence from 3 individual matrices A,B,C to long row includes elements of all 3 matrices together to ...
1
vote
1answer
10 views

Examples of “one to many” for RNN/LSTM

Are there any examples dealing with "one to many" kind of LSTM? Basically I am trying to build a model which takes an input vector $a$ and gives an output of $[b_1; b_2 ;b_3; b_4, \ldots; b_n]$ where ...
0
votes
0answers
21 views

LSTM - When to use sliding window in time series classification?

Say I have a tensor of data with shape (30, 16000, 38) - where each tuple element corresponds to ...
0
votes
0answers
3 views

when not using teacher forcing does backpropagation flow through input layers to output layers?

When training a Recurrent Neural Network that feeds outputs in as the next input(no teacher forcing) do gradients flow through input layers to output layers and also through the hidden states?
0
votes
0answers
25 views

Attention mechanism in LSTM model to predict next action

I have a dataset. Each data point of this set contains a variable length of sequences with 7 letters. For example, Data point 1 has a sequence (A, A, B, E, B, C, D, E, D.....). I used LSTM to predict ...
0
votes
0answers
10 views

How to make a sequence element-wise clustering with a RNN (preferable in Keras)

Non-Keras contributions are also welcome since the question is very concrete already. Imagine I have a sequence $S_i = s_0, s_1, ..., s_n$, where $s_k$ is the k-th element that represents an element ...
2
votes
0answers
17 views

LSTM Generates Duplicates

I'm using an LSTM to generate molecules using the Simplified Molecular Input Line Entry System (SMILES) to represent molecules. As an example, Aspirin is represented as O=C(C)Oc1ccccc1C(=O)O I have ...
0
votes
0answers
16 views

Why my LSTM model cannot predict waveform

Basically, I'm trying to use one waveform (above) to predict another waveform (). And Here are my model structure: ...
2
votes
2answers
78 views

How does an LSTM process sequences longer than its memory?

* Note: The premise of my question was incorrect in the first place. My question assumes that an LSTM maintains a separate set of weights for each time step in the memory it is given as a design ...
0
votes
1answer
25 views

How is Bidirectional-RNN different from vanilla RNN trained with both original & reverse copies of data?

I have several questions regarding Bi-RNN. The RNN here can be LSTM or GRU as well. (1) What is the input of Bi-RNN when making inference? For RNN, if I want to predict a $\hat{y}(t)$ for the target $...
0
votes
0answers
18 views

Binary target prediction using LSTM with sparse events in time

I have a data of patients that have multiple events happening in there medical history, I'd like to predict a target of having a specific targeted-event in the next 30 days. The data is timestamped ...
0
votes
1answer
48 views

How is the loss(Backpropagation) for simple RNN calculated when dealing with batch?

I have been trying to implement a simple RNN in Python. I saw Andrew Ng's course on RNNs, and then I tried to write one for myself. However, it seems I have not ...
0
votes
0answers
17 views

How to model a time series problem for RNNs?

I have projects as input data, each project has a weekly progress report (hours of work completed in this week). A project can have an arbitrary duration, but let's say it's usually around 100 weeks, ...
3
votes
1answer
127 views

Cannot understand LSTM inference

I seem to have stumbled on a hole in my understanding around LSTMs. In short, I cannot understand how even a simple one is actually fed samples, upon inference time/training time. Here are the details:...
2
votes
1answer
76 views

What happens to the initial hidden state in an RNN layer?

I thought I knew how RNNs work, however, when I tried to actually implement it myself, I faced some issues. For one, how do we deal with the initial hidden state? ...
0
votes
0answers
15 views

Small output range and delayed output? Predicting sine using LSTM

I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict 0.5*sin(t + N) given 0.5*sin(t) as an exercise. I have tweaked the model, changing the output layer ...
1
vote
0answers
33 views

Should I remove the trend from timeseries when using DeepAR

I saw that for some other algorithms for timeseries data it is advised to remove trend and seasonality before doing the prediction (ex: ARIMA and LSTM) I figured out from the paper that SageMaker's ...
0
votes
0answers
20 views

How to explain the decision rationale of time series data classification task using deep learning model

Is there a way to explain which part of the time series data the model is looking at in task that classifying time series data (e.g. video) by deep learning model? When deep learning model using RNN ...
2
votes
1answer
32 views

Machine learning for product names

I have a machine learning challenge I may be over thinking. I have a set of 3.5 million products (not unique, there are multiple instances of each product). Each product has a "description" from it's ...
0
votes
0answers
13 views

Is it possible to make LSTM model with 4dimension shape?

Hellow, wizards. I have time series data including sevaral days. I try to predict a grade of tomorrow, which is range from 0 to 100. And I assume that this grade depends on 3 time-series independent ...
0
votes
0answers
172 views

Number of Hidden Layer Nodes in Recurrent Neural Networks

There's already a decent discussion on how to select the right number of hidden layers and hidden nodes in a feed-forward neural network: How to choose the number of hidden layers and nodes in a ...
0
votes
0answers
98 views

Time-series classification of Kinect data using Keras

For my PhD project I recorded using Kinect and Myo 11 people performing Cardiopulmonary Resuscitation (CPR), repeatedly doing chest compressions to a manikin (one person per time). I collected in ...
2
votes
0answers
35 views

Why Create a Generator Manually for Neural Network when Keras has Built-in Generators? [closed]

So many tutorials I see have manually created generators in order to use the fit_generator() command in the model-fitting stage. However, these seem to exist already in Keras. E.g., ...
2
votes
0answers
48 views

Time Series Forecasting RNN: Difference Between Masking & Excluding Rows

Suppose you have missing values in a time series E.g. : t1 x1 y1 t2 ? ? t3 x3 y3 t4 ? ? t5 x5 y5 You are trying to forecast this time series using a recurrent ...
0
votes
1answer
25 views

How to train a RNN language model?

I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the ...
1
vote
1answer
76 views

What is the initial state of the tf.contrib.rnn.LSTMCell? [closed]

Does tf.contrib.rnn.LSTMCell assign itself an initial state of zeros or is it random for each batch or per complete run through (if I run the model twice will it have the same initial state both the ...
0
votes
1answer
20 views

How to understand “multimodal” RNNs for image captioning?

This paper Deep Visual-Semantic Alignments for Generating Image Descriptions on image captioning proposed a Multimodal Recurrent Neural Network architecture. From my understanding, the multimodal RNN ...
0
votes
0answers
27 views

What is the recommended maximum number of time steps for RNN or LSTM?

More time steps incurs longer training time, which above certain limit becomes impractical. What is the recommended maximum limit of the number of time steps for RNN or LSTM? I'm using a powerful ...
0
votes
1answer
29 views

image caption generator

I see two models of image caption generator online: In the above model, the first LSTM cell of decoder takes the entire image as an input. In the above model, all the LSTM cells of the decoder take ...
3
votes
0answers
20 views

How to incorporate uncertainty and noise information in training and prediction of neural networks?

I am trying to use RNNs to perform state estimation on noisy sensor data. The readings are from a GPS dataset and it provides $[longitude, latitude, n_{satellites}]$. The last column, which is the ...
0
votes
1answer
16 views

How does network structure (model complexity) affects covergence speed?

I trained Bi-GRU and HAN (Hierarchical Attention Networks) on my own datasets, and found HAN converges faster than Bi-GRU, within less number of epochs. What would be the reason for this? I guess ...
1
vote
1answer
22 views

how the long term memory works in an LTSM network

I am trying to understand how lstm networks work in particular the long term memory aspect. For example if we have a very long paragraph and at the start we have someone's name, Bob for example. Bob ...
2
votes
1answer
116 views

Selecting number of time lags for input in LSTM networks?

I know from theory that LSTM are meant to selectively capture long and short term dependencies in a sequence. I'm trying to implement LSTMs for a time series task and I notice that a lot of tutorials ...
0
votes
1answer
121 views

About RNN with variable length output vectors

I have several thousands samples with equal number of features (5000, they are time dependent) and I would like to predict of vectors with variable length. 1) I'm beginner in RNN, and I'd like to ...
1
vote
0answers
24 views

Deep Learning Variable Length Sequence Handling

I am trying to understand the best practice for handling different lengths of sequences in NLP tasks. Lets consider an example of convolution on sequences followed by max pool layer. We can handle ...
0
votes
0answers
27 views

Prepare data for stateful/stateless/return_sequences RNN

I'm trying this Tensorflow example that using GRU to predict and generate text. Suppose there is the text "Hello World yes", and the sequence length is 5. Then we prepare the training data to be (...
2
votes
1answer
284 views

How to deal with unknown classes with a neural network classifier?

I have a small RNN with a softmax output, which succesfully classifies sequences within a known set of n classes. Now I have the problem that there might be sequences in the test data which do not ...
0
votes
0answers
15 views

What is the best model for keyphrase extraction from super long text?

I’m working on a keyphrase extraction task. The biggest difficulty of this task is that the text is very long (5000-20000 words). I’ve tried several unsupervised algorithms such as Tf-idf and TextRank ...
3
votes
0answers
207 views

LSTM : multi-step multidimensional multivariate multi-site timeseries forecasting [closed]

I'm working on a project in which i'm trying to do a pollution forecasting. I googled around and found that LSTM is a good candidate for this task, however, I'm still struggling at how to adapt it to ...
2
votes
1answer
82 views

Forecasting algorithms for incomplete time series data [duplicate]

I want to forecast the demand of each SKU in my warehouse every week from the history transaction that I have collected. The data contains brand, product type, SKU, quantity, date(per day), price. But ...
0
votes
0answers
17 views

How to obtain embedded representation of single test instance after training

The first layer of my RNN is embedded layer as follows. ...
0
votes
0answers
29 views

State prediction of how long someone sleeps using neural nets

I have over hundred thousands of datapoints on how long individual people sleep. I also have information about how soft their beds are, their income, stress levels etc. At first I want to predicted ...
0
votes
1answer
23 views

How are RNNs with inputs greater than the defined sequence length implamented

To clarify the slightly ambiguous language in the title. I have an RNN (actually 2 stacked RNN layers) that take input X of size X [batch_size, sequence_length, features] the model is trying to ...
0
votes
1answer
174 views

LSTM output dimensionality

I am new to LSTMs. When reading the papers and websites about LSTM architecture, there is something I do not get. As I understand it, a single LSTM layer can have multiple LSTM cells (just like a ...
0
votes
1answer
94 views

Is this a sign of (bad) local minima encountered by this RNN?

On a binary classification task, I can get perfect performance on the training set, but no matter how strongly I regularize the recurrent neural network (dropout 0.99, L2 weight penalties of 0.01) the ...
1
vote
0answers
45 views

How to apply dropout in LSTMs?

Dropout in fully connected neural networks is simpl to visualize, by just 'dropping' connections between units with some probability set by hyperparamter p. However, how dropout works in recurrent ...
0
votes
0answers
30 views

Use of variable 'states' in a recurrent neural network in tensorflow? [duplicate]

The output of RNN implementations in Tensorflow is a tuple [outputs, states]. I am a bit confused about the use of the states variable. For example in this example of a simple LSTM implementation in ...
0
votes
0answers
32 views

Priming GRU in RNN: Are they doing it in the wrong way?

I am looking at some code (in PyTorch but the question is general) where they use a technique called "priming" in order to "start" the prediction of an RNN that mainly just consists of a single GRU (...
1
vote
1answer
1k views

Time steps in Keras LSTM

My understanding of time-series LSTM training is that the recurrent cell gets unrolled to a specified length (num_steps), and parameter updates are back-propagated ...