Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
500
questions
0
votes
0answers
7 views
RNN (LSTM) training on multiple time series
Regarding RNN training, We feed network a network -> point by point from the same time series (or image or smth else).
When we “switch from one time series to another”, what should be done or how ...
1
vote
0answers
16 views
Using LSTM to find mode of a sequence [on hold]
I trained an LSTM network to predict the mode of a sequence of real numbers. I found the performance to be poor. Initially, I thought this is a fairly easy objective and LSTM networks would perform ...
1
vote
0answers
13 views
Which deep learning model to use for sequence completion
I am trying to solve the problem of sequence completion.
Let's suppose we have ground truth sequence
(1,2,4,7,6,8,10,12,18,20)
The input to our model is an incomplete sequence. i.e (1,2,4, _ , _ ,_,...
1
vote
0answers
69 views
Does smoothed data work better for time series forecasting with LSTMs?
I am training a 3-layer LSTM on time series data ($10^6$ training samples) to predict the next point in the time series, where there is no seasonality and
the time series has been made stationary (...
0
votes
0answers
25 views
LSTM model for multistep univariate Time series forecasting
I have scenario where i have time series data (1 per day) for past 365 days. And I need to make a prediction for next 365 days. Is this possible using LSTM or any other ML models. I have been through ...
0
votes
1answer
32 views
How can RNN handle variable sized inputs? [duplicate]
I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? The answer here says RNN can handle variable ...
0
votes
0answers
18 views
How to normalize a dataset of multiple univariate time series with very different standard deviations?
I have the a data set representing the electricity consumption of 25 000 customer.
The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from ...
0
votes
0answers
18 views
LSTM cell state update equation wrong in Deep Learning book? (Equation 10.41)
In Equation 10.41 of the Deep Learning Book, the author writes the update equation of LSTM cell internal state as :
$$
s_{i}^{(t)}=f_{i}^{(t)} s_{i}^{(t-1)}+g_{i}^{(t)} \sigma\left(b_{i}+\sum_{j} U_{...
3
votes
1answer
38 views
Which machine learning model could be used for the following?
I am an experienced programmer but very new to machine learning.
I have a data set that consists of about 50,000 sets of 2,000 ordered values. All of the values are floats normalised to between 0 and ...
0
votes
0answers
13 views
LSTM Input data
I am really new to using LSTMs for time series forecasts. I am having a little problem defining the problem was hoping someone could help me.
Say I have a bunch of claim payments (see image). I ...
0
votes
1answer
32 views
Can we actually get the probability of a text using recurrent neural network?
I know that recurrent neural network is used to generate text and to model the probability of
$P(x_0,x_1,x_2,x_3)=P(x_0)P(x_1|x_0)P(x_2|x_1,x_0)P(x_3|x_2,x_1,x_0)$
where $x_i$ is words/text.
If RNN ...
0
votes
1answer
46 views
Are Epochs, Learning rate and Hidden units related to each other?
I'm working on an LSTM NN, I wanted to know if there is any relationship between the learning rate, epochs and the hidden units that will/might affect my classification output?
The MATLAB version I'...
0
votes
1answer
21 views
What is the minimum sequence length that can be used with the LSTM model for sequence classification?
the sequence is time-series data. It is possible to use as sequence length even low values such as 5 or 10?
Thank you
1
vote
0answers
30 views
How to train an LSTM model on multiple single-variable time-series data?
I am quite new to the field.
I am working on a problem involving time-series forecasting of single variable time-series. Data is collected from the pressure sensor on a patient in hospital.
Time ...
6
votes
1answer
21 views
Can I use a LSTM Autoencoder to compute similarity between two variable-length audio signals?
I would like to compute the similarity between audio signals of different length.
One way of doing it is to train a RNN (LSTM/GRU) Autoencoder and extract the hidden layer representation - feature ...
0
votes
0answers
14 views
Training an LSTM model for text summarization with different type of different domain data for training vs test
If I want to use an LSTM for doing text summarization but the labeled data I have (as in the summarizes which represent the labels) is from a different domain (Amazon reviews) but I have over 500K of ...
0
votes
0answers
16 views
Updating prediction with valid test data
The following scenario must be extremely common but I couldn't find a best practice for it readily available.
Suppose we require a predictive model (m) that is supposed to explain some variable y ...
0
votes
0answers
9 views
ConvLstm many-to-one model
I am trying to train a model using ConvLstm layers to learn the mapping between a sequence of 20 brain images (showing blood flow) of size 256x256, to a single blood flow parameter image of size ...
2
votes
1answer
40 views
Training a Neural Network on Temporal Image Data
I am working on a project where I have 1024x1024 brain images over time depicting blood flow. A blood flow parameter image is computed using the brain images over time, and is off the same dimension (...
0
votes
0answers
29 views
How can I reduce the noise of prediction graph? [duplicate]
I am trying to use LSTM to predict a time series data as you can see in the following image, the predicted graphs is very noisy:
The original data is looking like this:
That I normalized it like ...
0
votes
0answers
34 views
Multi-step ahead binary classification of multiple multivariate short time series
I'm working on a project where I need to identify loan defaults. I have around 50 000 time series, each time series represents a loan and is composed by few time steps (from 3 to 18). Each time step ...
0
votes
1answer
28 views
Can we use the folded version of RNN in training?
I always see that people teach the unfolded version of the RNN.
Do they do that just for the sake of teaching because the unfolded version is simple to explain sequence learning on it?
I read in an ...
0
votes
0answers
24 views
Does it make a difference in LSTM as to how I give the input?
So, as we know the input of the LSTM is always is a 3D array: batch_size, time_steps, seq_len. So, does it make a difference if I give input of the LSTM as: ...
0
votes
0answers
46 views
LSTM Regression - State of the Art
i'm trying to review the current state of the art architectures / models generally for Regression based on LSTM Networks (x-Sequence to y-Sequence / function approximation).
Basically, what are the ...
0
votes
1answer
45 views
LSTM Capacity: Many-to-Many vs. Many-to-One
Existing research documents LSTMs to perform poorly with timesteps > 1000 - i.e., inability to "remember" longer sequences. What's absent explicit mention is ...
0
votes
1answer
25 views
Is there any paper about vanishing-gradients of LSTM?
Some web pages mentioned that LSTM causes the vanishing or exploding gradients if the sequence is too long.
These are one of the pages mention the problem:
https://machinelearningmastery.com/...
2
votes
1answer
66 views
how to interpret the sharp decline in loss in seq2seq models
I have a seq2seq model. I have applied this data over 20_newsgroup data set.
My problem is that I face with exploding gradient ...
0
votes
0answers
23 views
How harmful is a wide Dense layer after a narrow?
My CNN-LSTM EEG Keras classification model includes a Dense 'shortcut' connection for residual sequence learning as shown below; to match dimensionality, the Dense layer's set to ...
0
votes
0answers
24 views
How can I interpret the result of get_weight of latent size in Seq2Seq model keras
My question is related to Seq2Seq models where we have LSTM as encoder and decoder.
Imagine we have the Autoencoder alone, and we extract the weight associated ...
0
votes
0answers
24 views
dimensionality reduction using SVD for forecasting with machine learning
I'm using a LSTM model to forecast time series data.
My dataset has far too many variables and I would like to perform dimensionality reduction. My LSTM model works on a rolling window of 500. I ...
0
votes
0answers
13 views
Time series classification - How much overlap?
I am trying to perform a sequence classification using LSTM. Let's say I have two time-series, series A and series B. Time series A has a length that is almost 100 times series B. I like to develop a ...
1
vote
0answers
59 views
Why do I get worse result after normalizing my input data of LSTM?
The first pic shows my data before normalization:
And the second pic show my data after normalization using sklearn.preprocessing.normalize :
I get better result ...
0
votes
0answers
20 views
How LSTM compare which information is important or not?
I am interested to know, if I have scaled my data between [0,1], and have a vector like [0, 0.001, 0.01, 0.1, 1], is that mean ...
0
votes
0answers
12 views
Constant output at each time step LSTM [duplicate]
I am facing a regression problem : Predicting electricity production using data from the last 3 hours to predict the next two.
I was using several continuous variables which were all standardized ...
0
votes
0answers
17 views
IP protocol flags prediction task - interpretation
I would like to create an algorithm able to predict the next flag in an IP protocol sequence. The sequences look similarly to this:
...
0
votes
2answers
38 views
Is Elmo equivalent to Fasttext+Bi-directional GRU?
From what I have read, Elmo uses bi-directional LSTM layers to give contextual embeddings for words in a sentence. So if I use a bi-directional LSTM/GRU layer over Fasttext representations of words, ...
2
votes
1answer
188 views
Are neural networks the wrong tool to solve this 2D platformer/shooter game? Is there a proven way to frame this problem to a neural network?
I had an attempt at training a convolutional LSTM network to imitate a player of the 2D platformer/shooter Teeworlds. This is an example of the data and labels I fed to the network:
https://www....
0
votes
0answers
102 views
Multiple Time series Forecasting Using LSTM in python
Assume I have a m dimensional input feature vector and I would like to perform multiple steps time series forecasting. I have about 500 files which each one is has 100 observations for example ...
0
votes
2answers
134 views
In an RNN, if the gradients don't vanish for long/distant terms, won't the derivative of the error be either divergent to infinity or oscillatory?
P.S. just cross posted here- https://datascience.stackexchange.com/questions/54322/in-an-rnn-if-the-gradients-dont-vanish-for-long-distant-terms-wont-the-deriv, as I still havne't got an answer from ...
0
votes
1answer
88 views
Trying to predict continuation of curves using LSTM
I have an application where I get a large set of smooth curves (2D). Those curves are represented by sample points on that curve. Sometimes, those curves cross or get close to each other and it is not ...
0
votes
0answers
50 views
Physics/dynamic system simulation with Deep Learning
I am interested in modelling physical/dynamic systems with deep learning, and I think that recurrent neural networks should be a good way to model systems that can be normally represented by state-...
0
votes
0answers
57 views
BiLSTMs with Attention model for Multi-Label Multi-Class Classification
I am trying the modify the BiLSTM with Attention model he used in Course 5 Neural Machine Translation for predicting grades (ranging from O,A+,A,B+,B,C,D,E,F) for multiple subject (approx 9 subjects ) ...
0
votes
0answers
22 views
Is negative Viterbi Loss possible?
So I'm training a sequence-labeling model with a BiLSTM-CRF architecture, and I am getting negative values on Viterbi Loss. Is this possible?
I'm using the following formula in my code, as specified ...
1
vote
1answer
26 views
How to infer state space parameters from an LSTM model?
I'm attempting to create a state-space model by training my time series data with an LSTM. I'm hoping the LSTM will capture non-linear phenomenon as opposed to a linear state-space model. The only ...
0
votes
0answers
69 views
Help understanding if suffering from Validation Bias
The goal is to forecast the volume a product will sell in future months.
There are about 107 products that are being bought by different customers for different uses.
It is univariate problem since ...
0
votes
0answers
7 views
Would it make sense to use an LSTM to predict a list of events that occur at a specific time
I have a dataset that for every hour of an entire month, contains a set of events that occur during the hour. These events are not predefined and can be any string, and are definitely not categorical ...
0
votes
0answers
62 views
How exactly keras LSTM layer works?
I try to create a sentiment analysis that have 7 classification. Let's say, I have 100.000 unique word (already converted into 100.000 integer) which have the longest input is 41. I created 3 layer ...
0
votes
1answer
32 views
Transformer based decoding
Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder ...
0
votes
0answers
38 views
LSTM Autoencoders - architecture
I am a bit confused about the structure of LSTM autoencoders, as far as I know, common way to construct vanilla autoencoders is bottleneck structure, for instance, start with 40 nodes, encode it to 30 ...
0
votes
0answers
15 views
What to use as a “stop” signal in a hierarchical LSTM model with continuous variable-length outputs
I am implementing a hierarchical sequence-to-sequence deep neural network model using long short-term memory (LSTM), where the bottom level of the hierarchy generates discrete outputs (characters from ...
