Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
770
questions
0
votes
0
answers
24
views
Understanding types of LSTM and their use cases
I'm currently considering to use RNN/LSTM for a predictive modelling project that involves time-variant points. From looking at the following types of LSTM/RNN (in the picture below), I want to try ...
0
votes
0
answers
17
views
Similar results between feedforward neural networks, recurrent neural network and LSTM for time series data - Is this standard?
Tl;dr: I have trained feedforward neural networks, recurrent neural networks and LSTM networks to predict behaviour of weather temperature. The results are almost all the same (see below). Is this ...
- 111
0
votes
0
answers
4
views
Question about understanding Weights of Keras LSTM model
I am implementing Federated Learning (FL) using Keras LSTM.
Starting with the simple example where multiple models are trained at different clients. Each client shares their model weights with the ...
- 382
0
votes
0
answers
21
views
Evaluation for LSTM model
I have created a model for text generation using LSTM. I am having chess sequences learned, reporting only the pieces moved during the moves. So when I move a pawn on my game there will be "p&...
0
votes
1
answer
31
views
Baseline model for predicting the load forecast
I have a model which uses the historical data to predict the electricity load consumption. I want to compare my model with a baseline model to show the performance, however I can not find a good base ...
- 13
0
votes
0
answers
9
views
How to increase accuracy and to receive better forecasts?
I am building a LSTM network and I am using the following structure:
...
- 11
0
votes
0
answers
19
views
The correct pre-processing of time-series data when using LSTM
I have a time-series data that I would like to use for forecasting the data trend using LSTM. I followed
https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-...
0
votes
0
answers
11
views
Predict gaps in time-series using LSTM
I have the number of timeseries with missing values (gaps). I want to train the LSTM NN for the prediction gaps task. Each time-series have the different numbers of gaps.
Now, I use the mean value ...
- 101
0
votes
0
answers
32
views
One anomaly detection model for all industries
Background - I'm creating a time-series anomaly detection (TSAD) model for the wifi throughput. My customers are 2 banks, 5 retail stores, 4 universities, 6 hospitals. Currently, I have 2 options to ...
1
vote
0
answers
23
views
Is it okay to use PCA then with LSTM
I am currently trying to see if there is any validity to this procedure:
Using PCA on data in tabular data
Then transforming the new "PCA data" into a multivariate time series
Using an LSTM ...
0
votes
1
answer
25
views
Keras LSTM: how is stated maintained within a single batch?
Assuming Keras LSTM will reset internal state after each batch, I want to understand how internal state is maintained within a single batch. Suppose then, batch_size=4, timesteps=3 and num_features=1. ...
- 23
0
votes
0
answers
8
views
Predicting the dates of intermittent events
Hello I am trying to predict the date and location of certain events using an LSTM model. The observations of dates are not structured (sometimes no events for many days and sometimes many events on ...
- 1
1
vote
0
answers
25
views
Deep learning for Non Image binary classification data
I am a beginner in Deep learning, I have a data set that contains patients' data (with categorical and numerical features (231 features in total)) and we need to classify if the patients have an "...
3
votes
0
answers
28
views
Stacking recurrent neural network layers horizontally vs vertically
In the top answer on this post: What are the advantages of stacking multiple LSTMs? the idea of stacking LSTMs vertically is distinguished from stacking
them horizontally. I quite don't understand ...
- 31
1
vote
1
answer
28
views
Multiple correlated multivariate time series
I have forecast dataset containing multiple multivariate time series that are not independent from each other. A state in one of the the series in time "t" can influence the state in another ...
0
votes
0
answers
31
views
How to predict next occurrence in sequence where data has both numerical and categorical values?
Lets say we have the following prediction problem.
Given data of the following form:
Number
HasProperty1
HasProperty2
1.5
0
1
1.5
0
1
2
1
0
2
1
0
Where ...
0
votes
1
answer
29
views
Multivariate time series forecasting and LSTM: When should I separate time series in different inputs
Let us suppose that I have a multivariate time series with two variables that vary together in time: var1 and var 2. And let us suppose that I want to forecast the n-ith value of var 2, by considering ...
- 398
0
votes
0
answers
13
views
How to relate a time varying variable to a specific date of interest?
I have a dataset where the date of interest is the date where a pipeline failure in a building occurs. Different buildings have different amounts of failures, hence different spacing in time between ...
0
votes
0
answers
7
views
Can Bidirectional LSTM under perform LSTM under the exact same setting?
I've been training an LSTM autoencoder for anomaly detection on Time Series.
I got some very satisfying results with this architecture and we were trying to improve the model performance.
We looked at ...
- 151
1
vote
2
answers
117
views
How to solve the problem of having sparse data that would become too small when aggregated?
I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
0
votes
0
answers
7
views
Return_State parameter in LSTM (Keras)
I'm a bit new to NLP, I've read multiple posts but couldn't find the intuitive purpose of setting return_state=True/False. I got answers like setting return_state will return cell output as well, this ...
- 101
0
votes
1
answer
297
views
How to inverse difference time series data?
I'm preparing a time series model with LSTM, I noticed that the time series data is not stationary so I used diff(period=1) in ...
1
vote
2
answers
67
views
Time series forecasting for unequal length data per year
I wanted to forecast student fall enrollment using last 3 years of data. Registration open day for fall differs by year. For instance, Fall registration open day in 2019 was March 10 and ended in say ...
- 131
0
votes
0
answers
13
views
Why does complex model like LSTM give lower accuracy than simple embedding on the validation set?
I was doing multi-class text classification task and I was doing comparison between LSTM and embedding and I found that simple embedding layer gives better accuracy than LSTM on the validation data. ...
- 1,250
1
vote
0
answers
74
views
What's the difference between stacked LSTM and encoder-decoder LSTM
I wanted to learn about encoder-decoder LSTM and after some digging around I get that the first LSTM layer in an encoder-decoder-LSTM outputs its hidden state and then the next LSTM layer uses that ...
0
votes
0
answers
8
views
Benefit of adding LSTM to CNN for video classification?
Regarding the accuracy result of the testing group (Like F1), when we try to do video classification (Like violent or not violent scene classification) , when we build a model the classification will ...
0
votes
0
answers
28
views
How does LSTM work during inference?
Suppose, I take a stock price prediction example.
And, let us assume that I am trying to predict "closing price" of stock based on "closing price" of past 10 days.
Now, I want to ...
0
votes
0
answers
13
views
How to compare different LSTM models with different features?
I have 3 LSTM models with different features.
LSTM1 is trained using Adjusted Close
LSTM2 is trained using Adjusted Close and <...
- 11
0
votes
0
answers
21
views
Compare three different algorithms for anomaly detection
I have 3 different anomaly detection algorithms, that I tested on a mock dataset of 5 elements.
The output of the first and second algorithms, that implement an LSTM, is true/false according to if ...
- 115
0
votes
1
answer
25
views
How do I obtain a percentage accuracy for an LSTM
I've got an LSTM trained for time series forecasting and I've seen people online report their LSTMs accuracy in %s such as 85% accurate etc, how do I obtain a metric like this? so far I was just using ...
1
vote
1
answer
367
views
Confused with binary cross-entropy vs categorical cross-entropy
I have a dataset with 10 input categorical features and one output categorical feature with class 0 and 1. X_train follows a 3D array so I have done label encoding beforehand on the dataset.
I have ...
- 13
0
votes
0
answers
22
views
Does adding dense layers after LSTM layer add to the vanishing gradient problem?
I want to implement the time-series prediction model using LSTM followed by Dense layers.
I was thinking that by adding multiple dense layers after the LSTM layer (see the diagram below), I'll get an ...
- 382
0
votes
0
answers
48
views
LSTM model can not predict properly but works on the test samples
I have an LSTM model as follow:
The problem is that:
It can work on the X_test.
However, when I choose a part of X_test and want to predict the next N ahead point, ...
- 101
0
votes
1
answer
29
views
Sequential recommendation: how to effective encoding output item?
Now I am learning about sequential recommendation - session based recommendation. I have understood that User-item interactions may be viewed as sequential action (first I clicked item A, then click ...
- 244
1
vote
0
answers
26
views
How to analyze data before building a timeseries forecasting model?
I want to build a timeseries forecasting model which uses LSTM layers, I want to know what analysis we have to do prior to fitting the model? My data is recorded every 20 minutes, I performed Dickey ...
1
vote
1
answer
145
views
Multiple time series forecasting: How to split the data for training of a neural network
Use case: I have sales of 90 products during the first 180 days since the product launch. I want to train an LSTM network to predict sales 4 weeks ahead given the last 7 days of sales. The model ...
- 111
1
vote
0
answers
75
views
How does Walk-Forward work with LSTM
I have been looking at how to split my data for training/validation/test for a timeseries using LSTM and have some conflicting thoughts I would like to get a bit more clarity on.
I came across: QA1 ...
- 41
1
vote
0
answers
21
views
References with mathematical formulation of the learning problem with neural networks [closed]
I'm a novice at using neural networks.
I've developed a good solution for a problem using a neural network that has two branches: a feed-forward network and a bi-LSTM, that receives different kinds of ...
- 398
0
votes
1
answer
104
views
Data input: Expanding or Sliding Windows for LSTMs?
External research R1 (Stock Prediction with ML: Walk-forward Modeling by Chad Gray on 18/07/2018 at alphascientist.com) led me to believe that a sliding window is
more favourable than an expanding ...
- 41
3
votes
1
answer
624
views
How to cross-validate a time series LSTM model?
I have been looking at how to split my data for training/validation/test for a timeseries using LSTM and came across: QA1 and QA2
Given I should implement walk-forward splits my depiction of it is:
...
- 41
1
vote
0
answers
15
views
What is possibly wrong in this Validation Accuracy and Training Accuracy [duplicate]
As general perception over training and validation accuracy is that if training accuracy is high and validation accuracy is marginally low, then it is most probably over fitting.
Consider a case of ...
0
votes
1
answer
39
views
Can I use test data in model to make predictions?
If I have say 1 year daily close price of a stock and I divide it in ratio of 80:20 as train:test data.
Now I use TimeSeriesGenerator to fit the model on train data.
After fitting the model I want to ...
- 117
1
vote
0
answers
16
views
When do I use h state or c state of LSTM?
I'm learning LSTM but I don't get when to use the h hidden/output state or the c carry/cell state. Some resources say the c ...
- 123
0
votes
1
answer
73
views
Which LSTM output should be used for predictions?
Using this question as background:
https://stackoverflow.com/questions/71023822/lstm-multi-variate-multi-feature-in-pytorch
I was wondering how one processes the output of a pytorch LSTM
I was using ...
- 3
0
votes
0
answers
29
views
LSTM Autoencoder for online anomaly detection
I would like to use an LSTM-Autoencoder for an anomaly detection task, but in an online setting, meaning we are observing the data as it is streaming in.
What I would like to do is given some discrete ...
- 135
0
votes
0
answers
20
views
How to prepare dataset for multivariate time series analysis using LSTM or RNNs?
I am looking to use LSTM/RNN to perform time series prediction using some clinical data, however the structuring of the data might be an issue. I am fairly new to LSTM/RNN, would like to know how can ...
- 1
2
votes
1
answer
308
views
Why not both standardize and normalize features for machine learning?
If one has data that's assumed to be normal distributed and want to use it as input in a machine learning model, why not first standardize the data and then normalize (min max scale it between zero ...
- 145
0
votes
2
answers
31
views
What degree of difference does validation and training loss need to have to be called overfit?
I've trained an LSTM network to predict time series data however i'm quite new to LSTMs and am unsure if the model has overfit.
I know that an increasing validation loss relative to a decreasing ...
0
votes
0
answers
71
views
Triplet loss for text embedding and text similarity? [duplicate]
I am working on a triplet loss based model for text embedding.
Short description:
I have a database about online shop, I need to find the suitble product when users enter a text on search bar. I ...
0
votes
0
answers
13
views
Calculating RNN loss (for a SINGLE example) as a sum of individual time step losses VS. an average of individual time step losses [duplicate]
In Andrew Ng's course, I see RNN loss being calculated as a sum of the losses from each time step as seen here:
In Stanford's CS224N, I see loss calculated as an average of individual losses as seen ...
- 1