Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
810
questions
0
votes
0
answers
8
views
LSTM validation accuracy fluctuating [duplicate]
I am using LSTM to model time series data. My target variable is categorical so I am using one-hot encoding. The goal is to predict the target class based on the given time. My dataset spans over ...
4
votes
2
answers
168
views
Model only exceeds baseline in a certain fine-tuning condition
I am training an vanilla 5-layers LSTM. My task is trying to compare two models between without (baseline) and with the additional features (compared model). However, I found out that the compared ...
- 43
0
votes
0
answers
24
views
Vanishing Gradient Problem: What is the cause from a Data perspective? [duplicate]
My question:
I know there exists a lot of information about what causes the vanishing gradient from a computational standpoint. Ie due to way the RNN is trained by backpropagation [...]. Why do RNNs ...
- 101
0
votes
0
answers
10
views
Why is the accuracy of a Decision Tree decreasing whereas the accuracy of an LSTM is increasing when adding augmented data?
I am using sklearn's DecisionTreeClassifier and LSTMs (Keras) for time series classification. To increase the accuracy and robustness of the models I augmented the ...
- 31
0
votes
0
answers
11
views
Memory Cells vs Attention: Learnable Parameters?
Are weights and other parameters inside memory cells such as LSTMs and GRUs learned during backprop?
If they do get optimized and learned, what is the difference between these cells and an attention ...
- 1,014
0
votes
0
answers
8
views
Reduce the error propagation in univariate dynamic multi-step forecasting using LSTM
I am trying to forecast a dynamic number of steps ahead depends on the user input. I tried to use a for loop but the error is increasing as the number of steps increase. I want to forecast up to 48 ...
- 1
0
votes
0
answers
24
views
LSTM accuracy not increasing more than 68%
I am a newbie in the field of deep learning. I am trying to use LSTM with time series data with more than 300,000 datapoints. The target variable is a class with 7 unique values. I used one-hot ...
- 13
0
votes
0
answers
25
views
Daily closing price predictions using LSTM
I am quite new to the theory of RNNs so please excuse me if the question is trivial. I'm trying to fit an LSTM model to predict daily closing prices of 10 stock. I'm planning to fit separate models to ...
1
vote
0
answers
23
views
Relative changes instead absolute values in LSTM training
I am reading some papers about glucose time series prediction and I have noticed that some of them propose LSTM models that use relative changes between two measures.
For example, if $$ glucose(t)=60, ...
- 11
0
votes
0
answers
16
views
comparing support vector regressor with lstm model using mlxtend
I wish to compare two ML models for regression tasks. I am using SVR and LSTM. When I use mlxtend's paired t-test it is throwing an error for estimator 2 which is the LSTM model. the shape of the ...
- 1
0
votes
0
answers
10
views
Seq2Seq LSTM or RNN with independent hidden state
I'm curious about sequence-to-sequence mapping using recurrence networks like LSTM and RNN. As I've seen so far, in machine translation using RNN or LSTM, people usually use an Encoder-Decoder network ...
0
votes
0
answers
7
views
How does LSTM perform inference on untrimmed video or audio?
I am new to the concept of RNN and LSTM. The concept of LSTM gave me an impression that it performs inference step by step. If the input data is a video, it first consumes Frame(t-1), and then ...
- 113
0
votes
0
answers
8
views
Text Classification model unable to learn [duplicate]
I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases.
I have researched for possible ...
- 1
1
vote
0
answers
21
views
How to decide column size and number of cells for LSTM
For the following Keras LSTM neural network, how to decide the number of columns/features for the input data set, the number of LSTM cells, and time steps?
...
- 111
1
vote
0
answers
22
views
Much better results when standardizing features to train LSTMs
I have a data set of time series. Each time series represents trajectories of the same path taken. So, the time series captures acceleration in $x$, $y$ and $z$ direction, respectively for the ...
- 11
1
vote
0
answers
14
views
Model Architecture Multi-Agents
My question is based on a previews post:
How to train LSTM model on multiple time series data? .
The task is basically the same. However, my question builds on that: What does the model architecture ...
- 11
0
votes
0
answers
43
views
Can LSTM be used to model categorical output?
I am trying to use an LSTM to model time series data. My dataset looks as follows with two float type columns and the index representing timestamp.
...
- 13
1
vote
1
answer
30
views
Time series forecasting for dataset spanning a week
I am a newbie in time series data forecasting. I have a week long data and the counts represent arrivals per 5 mins period. A part of the dataset is shown below.
...
- 13
0
votes
0
answers
23
views
How do I prepare data for a multivariate LSTM model that includes multiple patients
I want to predict the blood glucose levels using time series data with multiple features such as time, glucose levels, carbohydrates, fat, and protein. I have a dataset with hundreds of patients but ...
- 1
0
votes
0
answers
15
views
LSTM Hyperparameter tuning, reasonable range
As the title describes, I am not clear what a reasonable range for hyperparameter tuning is. I know that there is not a definite answer when it comes to hyperparameter tuning. However, I was wondering ...
- 635
0
votes
0
answers
14
views
Stateful LSTM and the right hidden state
When we are using stateful LSTMs, we don't reset the hidden state inbetween batches. Put differently, we use the final hidden state of the previous batch as the initial hidden state of the current ...
- 635
0
votes
0
answers
26
views
LSTMs: How to deal with the hidden state when predicting
I am trying to understand more about rnn's, more specific LSTM's. I am uncertain about some aspects. Let's consider the following example: We have a time-series with 100 elements. We use the first 70 ...
- 635
1
vote
0
answers
38
views
Why LSTM predicts very poorly with loss curves showing no indication of overfitting or underfitting?
I am training a LSTM model for load demand prediction with:
Training Data: 18288 samples with 9 features
Validation Data: 0.05% of Training Data (about 915 samples)
Data Scaling: MinMax Scaler(0,1)---&...
- 11
0
votes
0
answers
7
views
How to improve low magnitude and seasonality-missing predictions in LSTMs?
I am using DeepAR to predict count (small integers > 0) time series of Electric vehicle demand at a charging station.
The prediction output on the test set is as follows:
Blue is the actual and ...
- 221
0
votes
0
answers
58
views
Backpropagation through time formula on GRU
i searched the literature and found the backpropagation through time formula on LSTM as follows:
in the above formula, input gate, hidden state, output gate, input, cell state have derivative ...
- 47
2
votes
1
answer
81
views
Visualize LSTM for time series sequential data
I am trying to visualize LSTM that is applied to sequential data available here: https://its-ml.de/index.php/plattscaling/
There are 25 features with 50 sequence length.
LSTM layer 1 has 100 cells, ...
- 23
0
votes
0
answers
66
views
Batch size and epoch for LSTM [duplicate]
I am trying to learn LSTM that is applied to sequential data.
I am referring to following study material available here: https://its-ml.de/index.php/plattscaling/
Its given that:
sequence_length = 50
...
- 23
0
votes
1
answer
33
views
What degree of difference does validation and training loss need to have to be called good fit?
I am conducting a multi-variate time series forecasting using an LSTM model. The model architecture and other details are given below:
Dataset split: (80/10/10 split)
Training Data Points: 367640
...
- 11
0
votes
0
answers
16
views
Do I have to reset my lstm hidden state after each forward pass in reinforcment learning?
I want to make a model to learn a simple 2 player card game, where there is some randomness in the game, since a player can draw a random card. To, in a way, remember the cards drawn and what cards ...
- 1
0
votes
0
answers
19
views
Sequence Classification with LSTM - Instant overfitting and high variance in validation results. How to procede?
I am new here so please forgive me if I missed something in creating an good question. Just tell me and I will add missing information etc. :)
The task is to predict if a damage will occur with sensor ...
- 1
0
votes
0
answers
19
views
How to calculate "Param" values in LSTM [duplicate]
I am trying to learn LSTM that is applied to sequential data.
I am referring to following study material available here: https://its-ml.de/index.php/plattscaling/
Can somebody please let me know what ...
- 23
0
votes
0
answers
20
views
Return_state in LSTM Keras
Using Return_state = True, we return the last hidden state twice and the last cell state. My question is why the last hidden state is returned twice. Why is that needed and in which application?
- 97
1
vote
0
answers
124
views
LSTM from scratch
I already learned the basic model of LSTM, for example, layers, cell state, gates, and so on.
However, I am still slightly confused about how the whole thing is done; I want to simulate LSTM step-by-...
- 11
0
votes
0
answers
21
views
Why is the notion of a batch problematic for RNNs?
This paper says that the notion of a batch problematic for RNNs (page 9) (which is why you can't apply batch normalization for RNNs?). Why is it hard to talk about batches for RNNs?
Eg. the Pytorch ...
- 101
0
votes
0
answers
33
views
Why my lstm model does not overfit?
I have a time series of sensor data. I want to observe the overfitting phenomena. The amount of data:
...
- 141
1
vote
1
answer
36
views
Keras Prediction one step beyond
I am trying to make time series forecasting with keras. Has any one observe the phenomenon where the model can predict the next value after the current (the one that should predict)?
If in fact I move ...
- 21
0
votes
0
answers
48
views
How to interpret training and validation loss of DeepAR?
Please bear with me. Its a long but complete post.
My questions are:
Why does the training loss start to osccilate wildly after some epochs? It is because it has jumped out of a local minima? I tried ...
- 221
0
votes
1
answer
65
views
What deep learning methods should I explore for my panel data where one country is fixed?
The data looks like the following image and is available from 1980 to 2019:
As you can see, nation 1 doesn't differ in any of the panels; only country 2 does and the country 2s for each panel remain ...
- 1
1
vote
1
answer
36
views
Shape of tensorflow model input
I'm reading Masking and padding with Keras, in the beginning, an input example is:
...
- 2,173
0
votes
0
answers
58
views
LSTM/GRU and K-FoldCV
I am applying LSTM and GRU models to a financial problem where the dataset is a time series composed of 365 rows (365 continuous days) and 23 columns (my outcome is the closing price of a financial ...
- 121
0
votes
0
answers
43
views
What inference can be drawn from graph and underlying model
I am using lstm for time series prediction, here is graph of loss,accuracy, val_loss, accuracy,
Model is able to achieve following
1)Can someone share comments/observations/insights on graph and ...
3
votes
2
answers
530
views
What exactly is meant by isotropic and anisotropic with word vectors
From this paper https://aclanthology.org/D19-1006.pdf "How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings"
When they say word ...
- 142
0
votes
1
answer
31
views
Fitting an autoregressive models not using the classical Maximum Likelihood Estimator?
I know for example, one could try the Recurrent Neural Networks approaches. But are there any other options to these? I don't know if the questions is rather absurd.
I am thinking of models like ...
2
votes
0
answers
39
views
Can recurrent dropout be used as a Bayesian approximation?
To express model uncertainty in LSTM, we can use dropout as a Bayesian approximation for Gaussian model according to this, (dropout is kept during inference and that will result in a distribution of ...
1
vote
0
answers
25
views
RNN: How to feed multiple different time series to the network?
I'm new to RNN's and asking for advice on how to feed data to an RNN such as a LSTM.
I'm trying to forecast wind speeds at one location. For this location i have historical values as a time series. ...
- 33
0
votes
1
answer
36
views
Understanding train vs validation loss chart
I am training an LSTM to a univariate time series and I have some questions about how to evaluate the train vs validations loss charts and which number of epochs to use in the model.
To give more ...
- 371
0
votes
0
answers
22
views
How to deal with multivariate time series forecasting with neural networks, considering known values in the future for some of them?
I would like to deal with a time series forecasting problem where we have a multivariate time series, involving 5 different time series (4 inputs and 1 output). That is, at each time point we have 5 ...
- 658
1
vote
0
answers
18
views
How can I call a point in the future of a time series that I would like to forecast?
I'm a beginner in time series forecasting and I'm not sure if I understood correctly the meaning of forecast horizon. I was considering horizons as points in the future for which we would like to ...
- 658
0
votes
0
answers
25
views
Convolutional-LSTM model predicts same image every timestep
So i am training a convolutional-lstm model which uses VGG19 as the encoder model, which encodes a sequence of images and passes this sequences to a lstm which is then fed into a fully connected layer ...
- 1
0
votes
1
answer
207
views
Understanding the loss funcion in DeepAR
The loss function looks like below, where N is the number of time series. ignore N for now and say N = 1. T is length of prediction horizon.
During training $t_0$ starts from encoder time step 0 all ...
- 221