Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
618
questions
0
votes
0answers
10 views
LSTM versus Random forest for Time series forecasting
Would you use the same feature vector when forecasting with a LSTMN as a Random Forest? Say features like 'day of week' and 'hour'. Or does the LSTMN learn this by just remembering from previous time ...
1
vote
0answers
22 views
Time series dynamic model: can we also learn time varying models?
I would like to train multivariate time series models with time varying weight information (time-varying relationship between data and labels). My understanding is that for example, autoregressive ...
2
votes
1answer
22 views
Is randomly split dataset to train and test in lstm model reasonable?(human activity recognition)
I have build an lstm model to predict human activity recognition with dataset OPPORTUNITY. I did two experiment with different oder of processing as below,
normalized the dataset with minmax scaler, ...
0
votes
0answers
16 views
Test scores are way lower than cross-validation scores
I split my Dataset with 80% of the data for training and 20% for the test in the context of a binary classification task with a very unbalanced dataset.
On the training set I do a 3 folds ...
1
vote
1answer
31 views
LSTM performs poorly with monotonically increasing test set values never seen in training. Why?
I have a dataset of approximately monotonically increasing values (in a time-series). I am using keras and LSTM to train the ...
0
votes
0answers
12 views
LSTM Neural Network gets stuck in a specific state when trying to predict new states over many time periods
I have built an LSTM neural network for category, or latent state, prediction.
The data is more or less of the form:
x1 = continuos number of current record
x2 = continuous number of current record
x3 ...
0
votes
0answers
12 views
MultiVariate Multi Step forecasting , If I try to model this as regression how can I do it
I am working on project where I want to predict how much customer use my services which translates to Dollar amount. I have data which is having 500 Customers whose monthly usage of services need to ...
-1
votes
0answers
23 views
unbalanced data set how to change the objective function [closed]
I created my first LSTM model and it ran very nice and gave me a good starting point to learn more. The data set was well balanced.
I have another data set which is unbalanced. Say an event occurs ...
0
votes
1answer
22 views
Combining two sequences for text classification
I'm doing text classification on comments posted on articles/stories. The two human-labeled classes are appropriate and not appropriate (not the same as happy/angry or any "sentiment" ...
35
votes
4answers
5k views
Is LSTM (Long Short-Term Memory) dead?
From my own experience, LSTM has a long training time, and does not improve performance significantly in many real world tasks.
To make the question more specific, I want to ask when LSTM will work ...
2
votes
1answer
51 views
Weird behaviour in toy RNN (Keras, LSTM)
I'm trying to learn more about RNNs and I'm tackling a toy problem. I'm generating some data that has a pattern, two 1s followed by three 0s which keeps repeating infinitely without any noise. So my ...
1
vote
1answer
18 views
Understanding the output layer formation of an LSTM unit in Keras
I'm struggling to get my head around how the output shape of an LSTM layer formed. How is the output unit value physically implemented in the layer?
For example, if I have an input shape of (128 x 6) ...
0
votes
0answers
19 views
Why is my training unstable?
I am training a Variational autoencoder with and without data labels. When I use labels (blue line), validation error decreases with epochs but without labels (orange line) the training is unstable. ...
1
vote
1answer
30 views
I am unable to understand how LSTM is taking my data as input [closed]
X_train.shape, X_test.shape, Y_train.shape, Y_test.shape
((2457, 55, 26), (820, 55, 26), (2457, 3), (820, 3))
X_train is like having 2457 samples of matrix of size ...
1
vote
1answer
44 views
Are RNNs Markovian?
On the one hand, one can argue that they are since "the hidden layer is simply [derived from] the last hidden state and current input". On the other hand, the whole point of RNNs is that &...
0
votes
0answers
6 views
Keras LSTM POS tagger w/ transfer learning (GloVe) — failing to learn?
I've been trying to research how to use Keras to train a POS tagger; specifically I want it to use an LSTM architecture and to use word embeddings, namely, GloVe. I've taken inspiration from two blogs....
0
votes
0answers
10 views
Training an LSTM on multiple distinct batches of time series data
I am running a time series simulation on an electricity power grid simulation package and I want to use this data to train an LSTM to predict the stability of the grid over a given time interval.
My ...
0
votes
0answers
13 views
size of LSTM layer and regression on fraction
I need to give prediction on a variable with a large range of values (all positives).
I scaled the values between 0 and 1.
My first layer is an embedding layer, which it's vocabulary size might change....
0
votes
0answers
16 views
Forecasting sales for thousands of stores individually with multiple features associated
I have data of 2000 stores with associated 145 features(example: ambience, holidays, no. of brands) and their monthly sales for 2 years. It means that for every store I have sales data and other ...
0
votes
1answer
13 views
Why do Dense layers perform better than a mix of Conv Layers, Recurrent Layers on Sentiment Analysis with BERT emebddings?
I have used BERT to make embeddings out of the imdb review dataset and I am trying out some models to check their perfomance on sentiment analysis (0 for the bad reviews and 1 for the good ones). I ...
0
votes
0answers
8 views
Single LSTM, Multiple Output Layers with Different Loss Functions
Would it be possible to have an LSTM that is followed by two output layers, where each output layer computes a different representation and is followed by two different loss functions (i.e. where the ...
2
votes
1answer
20 views
Question about using rolling windows for time series regression
I have say 10 time series which become the 10 features of my model and I train it on these using a rolling window of 6 to predict the following 1 timestep (so t-5 to t to predict t+1). Thus the input ...
0
votes
0answers
15 views
What could explain a loss geting very small quickly in a LSTM network?
I am trying my first LSTM with keras to classify time-dependent data sets.
I have created a training and a testing data sets, which I have normalized:
...
2
votes
1answer
21 views
What is the use of the hidden state in an LSTM network
I am training an LSTM network for time series prediction. My understanding so far is that an LSTM network is suitable for time series prediction because it keeps a 'hidden state' which gives the LSTM ...
0
votes
0answers
21 views
Use deep learning to predict the continuation of one time series given another
I have this problem I'm trying to solve..
I have 2 highly correlated time series (lets call them $A$ & $B$), however past a certain date I only have data for A. I would like to use $A$ to predict ...
0
votes
0answers
12 views
Should LSTM data be a sequence?
let me explain what I want to do,
I want to predict the trend of the price of something (1 if it increases in the next hour and 0 otherwise). I have gathered tweets about that and grouped them in ...
0
votes
0answers
19 views
Is it possible to use an LSTM for time series classification?
I've read a lot of literature on using LSTM's for time-series prediction in the regression sense; using past values to predict the next value in a time series.[1][2]
However, I have not come across ...
0
votes
0answers
15 views
How exactly does conv1d filter work when operating on a sequence of characters?
I understand convolution filters when applied to an image (e.g. an 224x224 image with 3 in-channels transformed by 56 total filters of 5x5 conv to a 224x224 image with 56 out-channels). The key is ...
1
vote
0answers
16 views
Training and Testing LSTM [closed]
I've been reading this article about multivariate LSTM. I'm still a beginner so my questions might be silly.
As far as I understand, we need to divide the data set on training and testing/validation. ...
0
votes
1answer
61 views
When computing parameters, why is dimensions of hidden-output state of an LSTM-cell assumed same as the number of LSTM-cell?
I was trying to figure out how to estimate the number of parameters in an LSTM layer. What is the relationship of number of parameters with the num lstm-cells, input-dimension, and hidden output-state ...
1
vote
0answers
21 views
Multivariate time series classification/event detection
I have a 1.5 million row mutivariate time series dataset that looks like this:
...
1
vote
0answers
9 views
Scaling test set based on training will cause test set to have values greater than the scale
I have a time series data that does not have an upper limit (data is somewhat monotonically increasing). Making the Test set values larger than the training set. (I am not shuffling because time ...
0
votes
0answers
6 views
Large amount of missing values in as input features for LSTM time series
I am using an LSTM to predict a time series chart from multiple other time series charts as input features. The problem is that some of these input charts have much ...
0
votes
0answers
5 views
How to feed LSTM input correctly?
I have a time series problem with 15 minutes as a timestep.The complete data will be from 2016-09-01 00:00:15 to 2016-12-31 23:45:00.
I have 5 variables(v1,v2,v3,v4,v5) in the data frame and I want ...
0
votes
0answers
12 views
Number of bidirectional LSTMs in encoder-decoder model must equal the maximum length of input text/characters?
I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/.
It ...
1
vote
1answer
28 views
How are text-to-speech systems' spectrogram frames aligned for loss calculation?
A key aspect of how text-to-speech (TTS) machine-learning works is very unclear to me even after reading the Tacotron-2 paper and the Google AI blog.
https://ai.googleblog.com/2017/12/tacotron-2-...
0
votes
1answer
32 views
Number of parameters in an LSTM cell
I am not sure how to calculate my LSTM weights based on this link
and my Keras programing as below:
...
0
votes
0answers
14 views
Time series forecasting LSTM small dataset
I'm new to machine learning, i'm trying to use LSTM to forecast the power production of a solar power plant, i have a small dateset that contains 7200 rows and 4 columns, i divided the data into ...
1
vote
0answers
36 views
What are the differences between GARCH/ARCH and LSTM for time series prediction? [closed]
Can somebody explain in-detailed differences between the GARCH/ARCH model and LSTM for time-series prediction and how the model works under the hood?
When would one use GARCH/ARCH over LSTM?
2
votes
1answer
22 views
Is it possible to overfitting within single epoch
Let me put my question first. For a time-series prediciton, is it possible to overfit even within the first epoch, when training data and validation data should all "new" to model?
Features and ...
0
votes
0answers
12 views
how many spectogram frames per input character does text-to-speech (TTS) system Tacotron-2 generate?
I've been reading on Tacotron-2, a text-to-speech system, that generates speech just-like humans (indistinguisahble from humans) using the github https://github.com/Rayhane-mamah/Tacotron-2.
I'm very ...
0
votes
0answers
8 views
Train one model across number of the datasets with multiple features time series of diffrent duration, using categorical metadata
I'm trying to create model for prediction multiple correlated time series features.
Issue is that input dataset consists of a number of "projects" with different duration and different categorical ...
0
votes
1answer
35 views
LSTM network window size selection and effect
When working with an LSTM network in Keras. The first layer has the input_shape parameter show below.
model.add(LSTM(50, input_shape=(window_size, num_features), return_sequences=True))
I don't ...
1
vote
1answer
23 views
How to specify features that are common to all timesteps in a keras LSTM Model?
I am trying to build an LSTM model to predict temperature for a given day using say past 7 days of temperature, rainfall etc of a Zipcode or PinCode. I understand that the training dataset needs to be ...
0
votes
0answers
30 views
Why does validation loss so much higher than training?
I am trainning a LSTM model that seems to do well on the training set. However, for some reason, after the first few iterations, the validation loss shoots up significantly. I am not even how a loss ...
0
votes
0answers
13 views
What is a good PR-AUC and should I undersample time series for rare event detection? [duplicate]
I have a binary classifier for a highly imbalanced multivariate time series.
I use an LSTM Network to predict the next time step and use the prediction error to decide whether a data point is an ...
0
votes
0answers
9 views
introducing lag variables versus using RNNs in time series prediction
Just wondering, is there a fundamental difference between introducing lagged IVs during data preparation and then using standard statistical/machine learning models versus the uses of RNNs such as ...
1
vote
0answers
15 views
Should I re-standardize data when updating LSTM model with new data?
I'm fitting an LSTM for timeseries data and I'm hoping to train it "dynamically"āe.g. train it initially and then re-train/update with the next timestep when I get that data. Right now I'm fitting the ...
0
votes
2answers
85 views
What is a multilayer LSTM?
Apologies, this question is quite long.
I am trying to implement a paper on optimising the working of multilayer LSTM. The optimisation process works as follows:
First I wrote a sequential code for ...
0
votes
0answers
13 views
Is sampling the training set for hyperparameter optimisation a good speed up solution?
I am using Bayesian hyperparameter optimization for LSTM hyperparameters. This manages to find an optimum set of hyperparameters in much fewer iterations than a grid search.
My problem is that I ...
