Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
720
questions
0
votes
1answer
46 views
Fitting a LSTM for stock price prediction using industry sector data
I am quite new to the theory of RNNs so please excuse me if the question is trivial. I am trying to fit a multivariate LSTM neural network to predict stock prices from a firm in the S&P 500 list. ...
4
votes
2answers
80 views
How to predict with a stateful LSTM the next values
I trained a RNN using LSTM cells. I would like to make a predictions for the next 14 days. I get results that are plausible but after reading various blogs I'm not so sure if I'm doing the right thing....
0
votes
1answer
7 views
Adding features to Sequences for Dense or LSTM
I am confused about the best way to add features to a CNN or LSTM model. Say I have input features where each example is an array of len 10: say [3, 5, 8, 9, 1, 7, 44, 12, 11, 6] and this goes in an ...
0
votes
0answers
16 views
Getting better results on LSTM (Tensorflow) shaping timesteps into features (?)
I am building multiple models for several different environments (similar applications) with LSTMs on Tensorflow. I know and I have seen people commenting on how inappropriate it is to build LSTMs ...
1
vote
0answers
7 views
When should RNNs be used, seeing as they take much longer to train?
RNNs seem to take much longer to train in most if not all cases. I assume this is because the number of operations involved in training an RNN scales not only with the number of examples being fed ...
1
vote
0answers
10 views
In LSTM texts, sometimes the current input and previous hidden state are concatenated, then multiplied by a weight. Sometimes, the other way. Why?
In the first case, the input and previous hidden state are concatenated then multiplied by a weight matrix, resulting in four matrix multiplications.
e.g. https://bond-kirill-alexandrovich.medium.com/...
0
votes
0answers
15 views
Why is my LSTM for time series generating very similar results for multi step predictions?
I have created an LSTM for predicting future steps for time series data. I have trained on 70% of the data (9194) and done the testing on 30% (3941) of the data. I am training the LSTM using 30 ...
0
votes
0answers
6 views
Why loss function of my LSTM model drops a lot after training using the first 100 batches and then almost doesn't decrease?
I am training a LSTM model with one layer and 32 hidden neurons in tensorflow using tf.keras.layers.LSTMCell. The problem is a binary classification and the loss ...
0
votes
0answers
18 views
Data Leaking through convergence paramters in LSTMs
Procedure:
I choose a basic LSTM network architecture motivated by a time series classification problem and plan on not changing the architecture subsequently (no hyperparameter tuning).
Then I use ...
0
votes
1answer
24 views
Good architecture/approach for encoding text
I want to create a model that efficiently encodes text for retrieving images that match the description given in the text.
I have extracted features of images through VGG19 model(4096 features for ...
0
votes
0answers
20 views
Grouped Time Series forecasting with scikit-hts
I am trying to forecast sales for multiple time series I took from kaggle's Store item demand forecasting challenge. It consists of a long format time series for 10 stores and 50 items resulting in ...
0
votes
0answers
11 views
Conceptual question - is it correct to use categorical variables such as day, month, year as a fixed sequence input in LSTM?
I am working on a problem where I have to try to predict the dependent variable (continuous) every hour based on hourly temperature (the single continuous variable in predictor space), along with 4 ...
0
votes
0answers
33 views
Is GRU the minimum/simplest RNN to prevent vanishing or exploding?
I learned from this answer and this post that the forget gate in LSTM controls which information to vanish and which not, but I wonder if the LSTM or GRU is the minimum/simplest RNN to accomplish that(...
0
votes
0answers
19 views
How to compare SARIMA to LSTM?
I have a univariate time series which I want to fit a traditional SARIMA model and a LSTM neural network. After that, I want to compare both results to check which one is the best for my case. I am ...
0
votes
0answers
22 views
How to extract skills from job description using neural network
I am doing a project where I have to extract skills from Job Description.
I have attempted by cleaning data(not removing stopwords), applying POS tag, labelling sentences as skill/not_skill, trained ...
0
votes
0answers
12 views
Model predicting extremely different output for examples outside the dataset [duplicate]
I'm training a neural machine translator using the encoder-decoder approach and I've got around 95% validation accuracy and 0.4 validation loss. The model is translating correctly for most of the ...
0
votes
0answers
18 views
Recommendation for prediction of multivariate time series
I am looking for a recommendation (concrete package/framework/approach etc., preferably in Python/Keras or R as that is where I have experience) for predicting multivariate time series.
I do have ...
0
votes
0answers
15 views
Scaling multi input LSTM
I have a single layer LSTM model with 300 time series which try to predict the next value for one time series, based on past 12 values of the 300 time series. 56 is the number of slices of length 12 ...
0
votes
0answers
21 views
ML algorithm for high dimensional time series forecasting
I'm trying to make a forecasting model for goods prices in an economy (trying to forecast inflation).
Dataset: has 300 goods prices % monthly variations for last 6 years. And also added $n$ ...
0
votes
0answers
12 views
Machine Learning Model Not Learning [duplicate]
I'm trying to build a LSTM model that takes in 150 consecutive candlesticks' open, high, low, close and EMA indicator values (150, 5), then predicts whether price moves 20 points up or down first.
<...
0
votes
0answers
29 views
What is the best way to choose timesteps in LSTM for time series/forecasting data
I am developing a time-series prediction model with LSTM, as in LSTM model we need to define some timesteps so that model takes into account that many numbers of the previous day's data for the future ...
0
votes
0answers
13 views
Attention Scores in Seq2seq sorting example
This question is about the interpretation of the additive attention (Bahdanau) scores in a seq2seq problem concerning the sorting of an input numerical sequence.
I have a LSTM Encoder/Decoder model ...
0
votes
1answer
35 views
How do I prepare clinical data for multivariate time series analysis using LSTM or RNNs?
I am trying to predict the progression of disease using certain clinical data (time series data) and covariates (such as age, sex, race etc.). I am aware of the existence of mainstream machine ...
0
votes
0answers
10 views
LSTM batch error in validation vs error in whole validation
I am training an LSTM for different series (passing them all at once).
While training I print the MAE of each training batch and validation batch. At the end of the epoch, I print the "epoch ...
2
votes
0answers
31 views
How to preprocess binary labeled sequences of ordered in time data points to fit into RNN?
I am observing some repeated events which are guaranteed to give a binary outcome on termination and I am doing some data preparation so I can fit it into a model.
Each event has its own properties ...
1
vote
1answer
39 views
Why LSTM still works with only 1 time step
I'm new to ML and learning LSTM with this tutorial
https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/.
Scroll down to part "LSTM Network for ...
1
vote
0answers
26 views
How to implement Leave One Out Cross Validation for an LSTM Model
I am trying to implement a leave one out cross-validation for my time series LSTM model, but I am not sure how to go about it considering my dataset.
My dataset consists of flight IDs (1-279) which ...
0
votes
0answers
21 views
Understanding bidirectional LSTM in Keras for time series prediction
I am trying to implement a bidirectional LSTM in Keras. I have used a TimeSeries generator that takes 24 timesteps of a feature vector (from t0 to t23) and outputs a single prediction vector at ...
0
votes
0answers
16 views
How do LSTMs understand correlations between features?
Let's say I want to create a model that predicts the stock prices of 2 stocks, for each stock I have additional information such as the volume.
So the data I have would be Stock1Price, Stock1Volume, ...
1
vote
0answers
22 views
Correct way of mean-centering and scaling time series data used as input to an LSTM?
I am training an LSTM network in tensorflow/Keras which takes eight different time series/features as input. The input matrix given to the network has the form ( nSamples x nTimesteps x nFeatures ).
...
0
votes
0answers
19 views
Can't an LSTM count the number of cycles of the input?
Can an LSTM count number of cycles? Or, equivalently predict a linear trend from a sinusoidal input?
I have tried to train an LSTM network on a problem that I supposed would be easy for it.
The ...
0
votes
0answers
21 views
Handling dataset with lots of 0s and 1s [duplicate]
I am trying to minimize reconstruction loss of my dataset that have 74% boolean values of 0s and 1s. 32% of data is 0s and the rest 67% are 1s. Now, when I passed my dataset to autoencoder without ...
0
votes
0answers
22 views
LSTM data structure
I'm trying to predict trade international flows between countries using LSTM Neural Nets for which I have a panel structured data set with country-pair-year observations $x_{i, j, t}$. Given a ...
2
votes
0answers
33 views
How does LSTM/RNN deal with the correlated samples and the samples with different lengths
We know that LSTM/RNN has the following advantages compared with the basic machine learning algorithms:
Account the correlation between samples (usually the samples from time series are always ...
1
vote
1answer
22 views
multiple likely ys for one instance of x: word prediction with LSTM
I have a ML project that is about predicting (suggesting) the next word based on the last n words, using LSTM. The output is a softmax dense layer the size of the vocabulary that shows the probability ...
2
votes
1answer
110 views
probability distribution as output for my LSTM
I have been trying to make a language model that predict the next word, but with the assumption that there are multiple "correct" answers.
Input: dictionary indices + document topic data for ...
0
votes
0answers
19 views
How to deal with the loss exploding for LSTM regression task [duplicate]
I am training a LSTM for regression problems, but the loss function randomly shoots up as in the picture below:
I tried multiple things to prevent this, adjusting the learning rate, adjusting the ...
1
vote
1answer
25 views
Are LSTMs just RNNs that use a different activation function?
I read online that LSTMs are just RNNs that calculate neuron activations differently. I am not clear on the terminology though. Do we actually call these activations, because they refer not to the ...
0
votes
0answers
23 views
How to improve the prediction accuracy ofa LSTM model [duplicate]
I am trying to learning the LSTM method by training the model on a custom dataset. This dataset has 6000 vectors of size 7. The 85% of the 6000 vectors are used for the training and the rest for the ...
1
vote
1answer
49 views
Why do RNNs share weight?
If weights are not shared the number of parameters will be extremely large and difficult to compute which I understand.
I don't understand the argument that varying length inputs is taken care by ...
2
votes
0answers
60 views
LSTM backpropagation gradient regarding vanishing and exploding gradients problem
I was looking around for a good explanation as to why LSTMs are better able to handle vanishing and exploding gradients compared to vanilla RNNs. I know it is due to the cell memory $c_t$ acting as a ...
1
vote
0answers
20 views
LSTM predictions for time-series is both shifted and outputting wrong values
I am currently trying to apply an LSTM to a time-series problem with hourly data. I have 10 features that represent weather data (time-dependent), and I want to predict a target vector of solar ...
0
votes
0answers
43 views
Multi-output Neural Network only predicting one value
I have been using LSTM multi-output Neural Nets to perform two tasks, regression coupled with a classification. The data is in a time-series format where my dependent variable is trade quantity ...
1
vote
0answers
20 views
How can I use RNN to predict 5 sequence numbers based on historical data?
Supposing I want to predict the next 5 sequences of numbers based on the historical data. my training set is as follow:
historical data: 3634 38 51 190 127 2422 568 5796 578 60 1935 -> next_value: ...
2
votes
1answer
24 views
Choosing a clip gradient for LSTM (DeepAR)
I'm training DeepAR using the GluonTS library. I was getting NAN loss after some time of training (similar to this problem here: https://github.com/awslabs/gluon-ts/issues/833).
It worked for me to ...
0
votes
0answers
21 views
Why my LSTM model predictions is a straigth line?
I am newbie in neural networks and I am trying to build a LSTM model to predict future values. My problem is that the plot of predictions result returns a line in comparation with the testting data.
I ...
0
votes
0answers
25 views
How do you interpret LIME output?
I have an LSTM model doing multivariate time series regression. I have two predictions, one vastly higher than the other. I used the Python LIME package's RecurrentTabularExplainer to get this output. ...
1
vote
1answer
29 views
Neural network training better without MinMaxScaler()?
Forum,
I have a multivariate time series problem; For my master thesis I am investigating whether it is possible to forecast the movement direction of stock price with machine learning. My model looks ...
2
votes
1answer
47 views
Truncated data in Neural Network regression
I'm working on a Neural Network regression problem and the variable of interest (trade flows) that I'm trying to predict is skewed and truncated at 0.
I first tried to fit the model without prior ...
0
votes
1answer
27 views
parameters of bidirectional LSTM
I am new to LSTMs and also bidirectional lstms. I am trying to implement a model described in a scientific article. It says that the bilstm model has a layer size of 200 and number of hidden layers is ...
