The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [lstm]

A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

RNN (LSTM) training on multiple time series

Regarding RNN training, We feed network a network -> point by point from the same time series (or image or smth else). When we “switch from one time series to another”, what should be done or how ...
1
vote
0answers
16 views

Using LSTM to find mode of a sequence [on hold]

I trained an LSTM network to predict the mode of a sequence of real numbers. I found the performance to be poor. Initially, I thought this is a fairly easy objective and LSTM networks would perform ...
1
vote
0answers
13 views

Which deep learning model to use for sequence completion

I am trying to solve the problem of sequence completion. Let's suppose we have ground truth sequence (1,2,4,7,6,8,10,12,18,20) The input to our model is an incomplete sequence. i.e (1,2,4, _ , _ ,_,...
1
vote
0answers
69 views

Does smoothed data work better for time series forecasting with LSTMs?

I am training a 3-layer LSTM on time series data ($10^6$ training samples) to predict the next point in the time series, where there is no seasonality and the time series has been made stationary (...
0
votes
0answers
25 views

LSTM model for multistep univariate Time series forecasting

I have scenario where i have time series data (1 per day) for past 365 days. And I need to make a prediction for next 365 days. Is this possible using LSTM or any other ML models. I have been through ...
0
votes
1answer
32 views

How can RNN handle variable sized inputs? [duplicate]

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? The answer here says RNN can handle variable ...
0
votes
0answers
18 views

How to normalize a dataset of multiple univariate time series with very different standard deviations?

I have the a data set representing the electricity consumption of 25 000 customer. The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from ...
0
votes
0answers
18 views

LSTM cell state update equation wrong in Deep Learning book? (Equation 10.41)

In Equation 10.41 of the Deep Learning Book, the author writes the update equation of LSTM cell internal state as : $$ s_{i}^{(t)}=f_{i}^{(t)} s_{i}^{(t-1)}+g_{i}^{(t)} \sigma\left(b_{i}+\sum_{j} U_{...
3
votes
1answer
38 views

Which machine learning model could be used for the following?

I am an experienced programmer but very new to machine learning. I have a data set that consists of about 50,000 sets of 2,000 ordered values. All of the values are floats normalised to between 0 and ...
0
votes
0answers
13 views

LSTM Input data

I am really new to using LSTMs for time series forecasts. I am having a little problem defining the problem was hoping someone could help me. Say I have a bunch of claim payments (see image). I ...
0
votes
1answer
32 views

Can we actually get the probability of a text using recurrent neural network?

I know that recurrent neural network is used to generate text and to model the probability of $P(x_0,x_1,x_2,x_3)=P(x_0)P(x_1|x_0)P(x_2|x_1,x_0)P(x_3|x_2,x_1,x_0)$ where $x_i$ is words/text. If RNN ...
0
votes
1answer
46 views

Are Epochs, Learning rate and Hidden units related to each other?

I'm working on an LSTM NN, I wanted to know if there is any relationship between the learning rate, epochs and the hidden units that will/might affect my classification output? The MATLAB version I'...
0
votes
1answer
21 views

What is the minimum sequence length that can be used with the LSTM model for sequence classification?

the sequence is time-series data. It is possible to use as sequence length even low values such as 5 or 10? Thank you
1
vote
0answers
30 views

How to train an LSTM model on multiple single-variable time-series data?

I am quite new to the field. I am working on a problem involving time-series forecasting of single variable time-series. Data is collected from the pressure sensor on a patient in hospital. Time ...
6
votes
1answer
21 views

Can I use a LSTM Autoencoder to compute similarity between two variable-length audio signals?

I would like to compute the similarity between audio signals of different length. One way of doing it is to train a RNN (LSTM/GRU) Autoencoder and extract the hidden layer representation - feature ...
0
votes
0answers
14 views

Training an LSTM model for text summarization with different type of different domain data for training vs test

If I want to use an LSTM for doing text summarization but the labeled data I have (as in the summarizes which represent the labels) is from a different domain (Amazon reviews) but I have over 500K of ...
0
votes
0answers
16 views

Updating prediction with valid test data

The following scenario must be extremely common but I couldn't find a best practice for it readily available. Suppose we require a predictive model (m) that is supposed to explain some variable y ...
0
votes
0answers
9 views

ConvLstm many-to-one model

I am trying to train a model using ConvLstm layers to learn the mapping between a sequence of 20 brain images (showing blood flow) of size 256x256, to a single blood flow parameter image of size ...
2
votes
1answer
40 views

Training a Neural Network on Temporal Image Data

I am working on a project where I have 1024x1024 brain images over time depicting blood flow. A blood flow parameter image is computed using the brain images over time, and is off the same dimension (...
0
votes
0answers
29 views

How can I reduce the noise of prediction graph? [duplicate]

I am trying to use LSTM to predict a time series data as you can see in the following image, the predicted graphs is very noisy: The original data is looking like this: That I normalized it like ...
0
votes
0answers
34 views

Multi-step ahead binary classification of multiple multivariate short time series

I'm working on a project where I need to identify loan defaults. I have around 50 000 time series, each time series represents a loan and is composed by few time steps (from 3 to 18). Each time step ...
0
votes
1answer
28 views

Can we use the folded version of RNN in training?

I always see that people teach the unfolded version of the RNN. Do they do that just for the sake of teaching because the unfolded version is simple to explain sequence learning on it? I read in an ...
0
votes
0answers
24 views

Does it make a difference in LSTM as to how I give the input?

So, as we know the input of the LSTM is always is a 3D array: batch_size, time_steps, seq_len. So, does it make a difference if I give input of the LSTM as: ...
0
votes
0answers
46 views

LSTM Regression - State of the Art

i'm trying to review the current state of the art architectures / models generally for Regression based on LSTM Networks (x-Sequence to y-Sequence / function approximation). Basically, what are the ...
0
votes
1answer
45 views

LSTM Capacity: Many-to-Many vs. Many-to-One

Existing research documents LSTMs to perform poorly with timesteps > 1000 - i.e., inability to "remember" longer sequences. What's absent explicit mention is ...
0
votes
1answer
25 views

Is there any paper about vanishing-gradients of LSTM?

Some web pages mentioned that LSTM causes the vanishing or exploding gradients if the sequence is too long. These are one of the pages mention the problem: https://machinelearningmastery.com/...
2
votes
1answer
66 views

how to interpret the sharp decline in loss in seq2seq models

I have a seq2seq model. I have applied this data over 20_newsgroup data set. My problem is that I face with exploding gradient ...
0
votes
0answers
23 views

How harmful is a wide Dense layer after a narrow?

My CNN-LSTM EEG Keras classification model includes a Dense 'shortcut' connection for residual sequence learning as shown below; to match dimensionality, the Dense layer's set to ...
0
votes
0answers
24 views

How can I interpret the result of get_weight of latent size in Seq2Seq model keras

My question is related to Seq2Seq models where we have LSTM as encoder and decoder. Imagine we have the Autoencoder alone, and we extract the weight associated ...
0
votes
0answers
24 views

dimensionality reduction using SVD for forecasting with machine learning

I'm using a LSTM model to forecast time series data. My dataset has far too many variables and I would like to perform dimensionality reduction. My LSTM model works on a rolling window of 500. I ...
0
votes
0answers
13 views

Time series classification - How much overlap?

I am trying to perform a sequence classification using LSTM. Let's say I have two time-series, series A and series B. Time series A has a length that is almost 100 times series B. I like to develop a ...
1
vote
0answers
59 views

Why do I get worse result after normalizing my input data of LSTM?

The first pic shows my data before normalization: And the second pic show my data after normalization using sklearn.preprocessing.normalize : I get better result ...
0
votes
0answers
20 views

How LSTM compare which information is important or not?

I am interested to know, if I have scaled my data between [0,1], and have a vector like [0, 0.001, 0.01, 0.1, 1], is that mean ...
0
votes
0answers
12 views

Constant output at each time step LSTM [duplicate]

I am facing a regression problem : Predicting electricity production using data from the last 3 hours to predict the next two. I was using several continuous variables which were all standardized ...
0
votes
0answers
17 views

IP protocol flags prediction task - interpretation

I would like to create an algorithm able to predict the next flag in an IP protocol sequence. The sequences look similarly to this: ...
0
votes
2answers
38 views

Is Elmo equivalent to Fasttext+Bi-directional GRU?

From what I have read, Elmo uses bi-directional LSTM layers to give contextual embeddings for words in a sentence. So if I use a bi-directional LSTM/GRU layer over Fasttext representations of words, ...
2
votes
1answer
188 views

Are neural networks the wrong tool to solve this 2D platformer/shooter game? Is there a proven way to frame this problem to a neural network?

I had an attempt at training a convolutional LSTM network to imitate a player of the 2D platformer/shooter Teeworlds. This is an example of the data and labels I fed to the network: https://www....
0
votes
0answers
102 views

Multiple Time series Forecasting Using LSTM in python

Assume I have a m dimensional input feature vector and I would like to perform multiple steps time series forecasting. I have about 500 files which each one is has 100 observations for example ...
0
votes
2answers
134 views

In an RNN, if the gradients don't vanish for long/distant terms, won't the derivative of the error be either divergent to infinity or oscillatory?

P.S. just cross posted here- https://datascience.stackexchange.com/questions/54322/in-an-rnn-if-the-gradients-dont-vanish-for-long-distant-terms-wont-the-deriv, as I still havne't got an answer from ...
0
votes
1answer
88 views

Trying to predict continuation of curves using LSTM

I have an application where I get a large set of smooth curves (2D). Those curves are represented by sample points on that curve. Sometimes, those curves cross or get close to each other and it is not ...
0
votes
0answers
50 views

Physics/dynamic system simulation with Deep Learning

I am interested in modelling physical/dynamic systems with deep learning, and I think that recurrent neural networks should be a good way to model systems that can be normally represented by state-...
0
votes
0answers
57 views

BiLSTMs with Attention model for Multi-Label Multi-Class Classification

I am trying the modify the BiLSTM with Attention model he used in Course 5 Neural Machine Translation for predicting grades (ranging from O,A+,A,B+,B,C,D,E,F) for multiple subject (approx 9 subjects ) ...
0
votes
0answers
22 views

Is negative Viterbi Loss possible?

So I'm training a sequence-labeling model with a BiLSTM-CRF architecture, and I am getting negative values on Viterbi Loss. Is this possible? I'm using the following formula in my code, as specified ...
1
vote
1answer
26 views

How to infer state space parameters from an LSTM model?

I'm attempting to create a state-space model by training my time series data with an LSTM. I'm hoping the LSTM will capture non-linear phenomenon as opposed to a linear state-space model. The only ...
0
votes
0answers
69 views

Help understanding if suffering from Validation Bias

The goal is to forecast the volume a product will sell in future months. There are about 107 products that are being bought by different customers for different uses. It is univariate problem since ...
0
votes
0answers
7 views

Would it make sense to use an LSTM to predict a list of events that occur at a specific time

I have a dataset that for every hour of an entire month, contains a set of events that occur during the hour. These events are not predefined and can be any string, and are definitely not categorical ...
0
votes
0answers
62 views

How exactly keras LSTM layer works?

I try to create a sentiment analysis that have 7 classification. Let's say, I have 100.000 unique word (already converted into 100.000 integer) which have the longest input is 41. I created 3 layer ...
0
votes
1answer
32 views

Transformer based decoding

Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder ...
0
votes
0answers
38 views

LSTM Autoencoders - architecture

I am a bit confused about the structure of LSTM autoencoders, as far as I know, common way to construct vanilla autoencoders is bottleneck structure, for instance, start with 40 nodes, encode it to 30 ...
0
votes
0answers
15 views

What to use as a “stop” signal in a hierarchical LSTM model with continuous variable-length outputs

I am implementing a hierarchical sequence-to-sequence deep neural network model using long short-term memory (LSTM), where the bottom level of the hierarchy generates discrete outputs (characters from ...