Questions tagged [lstm]

A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.

Filter by
Sorted by
Tagged with
0 votes
0 answers
24 views

Understanding types of LSTM and their use cases

I'm currently considering to use RNN/LSTM for a predictive modelling project that involves time-variant points. From looking at the following types of LSTM/RNN (in the picture below), I want to try ...
user avatar
0 votes
0 answers
17 views

Similar results between feedforward neural networks, recurrent neural network and LSTM for time series data - Is this standard?

Tl;dr: I have trained feedforward neural networks, recurrent neural networks and LSTM networks to predict behaviour of weather temperature. The results are almost all the same (see below). Is this ...
user avatar
  • 111
0 votes
0 answers
4 views

Question about understanding Weights of Keras LSTM model

I am implementing Federated Learning (FL) using Keras LSTM. Starting with the simple example where multiple models are trained at different clients. Each client shares their model weights with the ...
user avatar
  • 382
0 votes
0 answers
21 views

Evaluation for LSTM model

I have created a model for text generation using LSTM. I am having chess sequences learned, reporting only the pieces moved during the moves. So when I move a pawn on my game there will be "p&...
user avatar
0 votes
1 answer
31 views

Baseline model for predicting the load forecast

I have a model which uses the historical data to predict the electricity load consumption. I want to compare my model with a baseline model to show the performance, however I can not find a good base ...
user avatar
  • 13
0 votes
0 answers
9 views

How to increase accuracy and to receive better forecasts?

I am building a LSTM network and I am using the following structure: ...
user avatar
0 votes
0 answers
19 views

The correct pre-processing of time-series data when using LSTM

I have a time-series data that I would like to use for forecasting the data trend using LSTM. I followed https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-...
user avatar
0 votes
0 answers
11 views

Predict gaps in time-series using LSTM

I have the number of timeseries with missing values (gaps). I want to train the LSTM NN for the prediction gaps task. Each time-series have the different numbers of gaps. Now, I use the mean value ...
user avatar
0 votes
0 answers
32 views

One anomaly detection model for all industries

Background - I'm creating a time-series anomaly detection (TSAD) model for the wifi throughput. My customers are 2 banks, 5 retail stores, 4 universities, 6 hospitals. Currently, I have 2 options to ...
user avatar
1 vote
0 answers
23 views

Is it okay to use PCA then with LSTM

I am currently trying to see if there is any validity to this procedure: Using PCA on data in tabular data Then transforming the new "PCA data" into a multivariate time series Using an LSTM ...
user avatar
0 votes
1 answer
25 views

Keras LSTM: how is stated maintained within a single batch?

Assuming Keras LSTM will reset internal state after each batch, I want to understand how internal state is maintained within a single batch. Suppose then, batch_size=4, timesteps=3 and num_features=1. ...
user avatar
  • 23
0 votes
0 answers
8 views

Predicting the dates of intermittent events

Hello I am trying to predict the date and location of certain events using an LSTM model. The observations of dates are not structured (sometimes no events for many days and sometimes many events on ...
user avatar
1 vote
0 answers
25 views

Deep learning for Non Image binary classification data

I am a beginner in Deep learning, I have a data set that contains patients' data (with categorical and numerical features (231 features in total)) and we need to classify if the patients have an "...
user avatar
3 votes
0 answers
28 views

Stacking recurrent neural network layers horizontally vs vertically

In the top answer on this post: What are the advantages of stacking multiple LSTMs? the idea of stacking LSTMs vertically is distinguished from stacking them horizontally. I quite don't understand ...
user avatar
  • 31
1 vote
1 answer
28 views

Multiple correlated multivariate time series

I have forecast dataset containing multiple multivariate time series that are not independent from each other. A state in one of the the series in time "t" can influence the state in another ...
user avatar
0 votes
0 answers
31 views

How to predict next occurrence in sequence where data has both numerical and categorical values?

Lets say we have the following prediction problem. Given data of the following form: Number HasProperty1 HasProperty2 1.5 0 1 1.5 0 1 2 1 0 2 1 0 Where ...
user avatar
0 votes
1 answer
29 views

Multivariate time series forecasting and LSTM: When should I separate time series in different inputs

Let us suppose that I have a multivariate time series with two variables that vary together in time: var1 and var 2. And let us suppose that I want to forecast the n-ith value of var 2, by considering ...
user avatar
  • 398
0 votes
0 answers
13 views

How to relate a time varying variable to a specific date of interest?

I have a dataset where the date of interest is the date where a pipeline failure in a building occurs. Different buildings have different amounts of failures, hence different spacing in time between ...
user avatar
0 votes
0 answers
7 views

Can Bidirectional LSTM under perform LSTM under the exact same setting?

I've been training an LSTM autoencoder for anomaly detection on Time Series. I got some very satisfying results with this architecture and we were trying to improve the model performance. We looked at ...
user avatar
1 vote
2 answers
117 views

How to solve the problem of having sparse data that would become too small when aggregated?

I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
user avatar
0 votes
0 answers
7 views

Return_State parameter in LSTM (Keras)

I'm a bit new to NLP, I've read multiple posts but couldn't find the intuitive purpose of setting return_state=True/False. I got answers like setting return_state will return cell output as well, this ...
user avatar
  • 101
0 votes
1 answer
297 views

How to inverse difference time series data?

I'm preparing a time series model with LSTM, I noticed that the time series data is not stationary so I used diff(period=1) in ...
user avatar
1 vote
2 answers
67 views

Time series forecasting for unequal length data per year

I wanted to forecast student fall enrollment using last 3 years of data. Registration open day for fall differs by year. For instance, Fall registration open day in 2019 was March 10 and ended in say ...
user avatar
0 votes
0 answers
13 views

Why does complex model like LSTM give lower accuracy than simple embedding on the validation set?

I was doing multi-class text classification task and I was doing comparison between LSTM and embedding and I found that simple embedding layer gives better accuracy than LSTM on the validation data. ...
user avatar
  • 1,250
1 vote
0 answers
74 views

What's the difference between stacked LSTM and encoder-decoder LSTM

I wanted to learn about encoder-decoder LSTM and after some digging around I get that the first LSTM layer in an encoder-decoder-LSTM outputs its hidden state and then the next LSTM layer uses that ...
user avatar
0 votes
0 answers
8 views

Benefit of adding LSTM to CNN for video classification?

Regarding the accuracy result of the testing group (Like F1), when we try to do video classification (Like violent or not violent scene classification) , when we build a model the classification will ...
user avatar
0 votes
0 answers
28 views

How does LSTM work during inference?

Suppose, I take a stock price prediction example. And, let us assume that I am trying to predict "closing price" of stock based on "closing price" of past 10 days. Now, I want to ...
user avatar
0 votes
0 answers
13 views

How to compare different LSTM models with different features?

I have 3 LSTM models with different features. LSTM1 is trained using Adjusted Close LSTM2 is trained using Adjusted Close and <...
user avatar
0 votes
0 answers
21 views

Compare three different algorithms for anomaly detection

I have 3 different anomaly detection algorithms, that I tested on a mock dataset of 5 elements. The output of the first and second algorithms, that implement an LSTM, is true/false according to if ...
user avatar
  • 115
0 votes
1 answer
25 views

How do I obtain a percentage accuracy for an LSTM

I've got an LSTM trained for time series forecasting and I've seen people online report their LSTMs accuracy in %s such as 85% accurate etc, how do I obtain a metric like this? so far I was just using ...
user avatar
1 vote
1 answer
367 views

Confused with binary cross-entropy vs categorical cross-entropy

I have a dataset with 10 input categorical features and one output categorical feature with class 0 and 1. X_train follows a 3D array so I have done label encoding beforehand on the dataset. I have ...
user avatar
  • 13
0 votes
0 answers
22 views

Does adding dense layers after LSTM layer add to the vanishing gradient problem?

I want to implement the time-series prediction model using LSTM followed by Dense layers. I was thinking that by adding multiple dense layers after the LSTM layer (see the diagram below), I'll get an ...
user avatar
  • 382
0 votes
0 answers
48 views

LSTM model can not predict properly but works on the test samples

I have an LSTM model as follow: The problem is that: It can work on the X_test. However, when I choose a part of X_test and want to predict the next N ahead point, ...
user avatar
  • 101
0 votes
1 answer
29 views

Sequential recommendation: how to effective encoding output item?

Now I am learning about sequential recommendation - session based recommendation. I have understood that User-item interactions may be viewed as sequential action (first I clicked item A, then click ...
user avatar
  • 244
1 vote
0 answers
26 views

How to analyze data before building a timeseries forecasting model?

I want to build a timeseries forecasting model which uses LSTM layers, I want to know what analysis we have to do prior to fitting the model? My data is recorded every 20 minutes, I performed Dickey ...
user avatar
1 vote
1 answer
145 views

Multiple time series forecasting: How to split the data for training of a neural network

Use case: I have sales of 90 products during the first 180 days since the product launch. I want to train an LSTM network to predict sales 4 weeks ahead given the last 7 days of sales. The model ...
user avatar
1 vote
0 answers
75 views

How does Walk-Forward work with LSTM

I have been looking at how to split my data for training/validation/test for a timeseries using LSTM and have some conflicting thoughts I would like to get a bit more clarity on. I came across: QA1 ...
user avatar
  • 41
1 vote
0 answers
21 views

References with mathematical formulation of the learning problem with neural networks [closed]

I'm a novice at using neural networks. I've developed a good solution for a problem using a neural network that has two branches: a feed-forward network and a bi-LSTM, that receives different kinds of ...
user avatar
  • 398
0 votes
1 answer
104 views

Data input: Expanding or Sliding Windows for LSTMs?

External research R1 (Stock Prediction with ML: Walk-forward Modeling by Chad Gray on 18/07/2018 at alphascientist.com) led me to believe that a sliding window is more favourable than an expanding ...
user avatar
  • 41
3 votes
1 answer
624 views

How to cross-validate a time series LSTM model?

I have been looking at how to split my data for training/validation/test for a timeseries using LSTM and came across: QA1 and QA2 Given I should implement walk-forward splits my depiction of it is: ...
user avatar
  • 41
1 vote
0 answers
15 views

What is possibly wrong in this Validation Accuracy and Training Accuracy [duplicate]

As general perception over training and validation accuracy is that if training accuracy is high and validation accuracy is marginally low, then it is most probably over fitting. Consider a case of ...
user avatar
0 votes
1 answer
39 views

Can I use test data in model to make predictions?

If I have say 1 year daily close price of a stock and I divide it in ratio of 80:20 as train:test data. Now I use TimeSeriesGenerator to fit the model on train data. After fitting the model I want to ...
user avatar
1 vote
0 answers
16 views

When do I use h state or c state of LSTM?

I'm learning LSTM but I don't get when to use the h hidden/output state or the c carry/cell state. Some resources say the c ...
user avatar
  • 123
0 votes
1 answer
73 views

Which LSTM output should be used for predictions?

Using this question as background: https://stackoverflow.com/questions/71023822/lstm-multi-variate-multi-feature-in-pytorch I was wondering how one processes the output of a pytorch LSTM I was using ...
user avatar
  • 3
0 votes
0 answers
29 views

LSTM Autoencoder for online anomaly detection

I would like to use an LSTM-Autoencoder for an anomaly detection task, but in an online setting, meaning we are observing the data as it is streaming in. What I would like to do is given some discrete ...
user avatar
0 votes
0 answers
20 views

How to prepare dataset for multivariate time series analysis using LSTM or RNNs?

I am looking to use LSTM/RNN to perform time series prediction using some clinical data, however the structuring of the data might be an issue. I am fairly new to LSTM/RNN, would like to know how can ...
user avatar
2 votes
1 answer
308 views

Why not both standardize and normalize features for machine learning?

If one has data that's assumed to be normal distributed and want to use it as input in a machine learning model, why not first standardize the data and then normalize (min max scale it between zero ...
user avatar
0 votes
2 answers
31 views

What degree of difference does validation and training loss need to have to be called overfit?

I've trained an LSTM network to predict time series data however i'm quite new to LSTMs and am unsure if the model has overfit. I know that an increasing validation loss relative to a decreasing ...
user avatar
0 votes
0 answers
71 views

Triplet loss for text embedding and text similarity? [duplicate]

I am working on a triplet loss based model for text embedding. Short description: I have a database about online shop, I need to find the suitble product when users enter a text on search bar. I ...
user avatar
0 votes
0 answers
13 views

Calculating RNN loss (for a SINGLE example) as a sum of individual time step losses VS. an average of individual time step losses [duplicate]

In Andrew Ng's course, I see RNN loss being calculated as a sum of the losses from each time step as seen here: In Stanford's CS224N, I see loss calculated as an average of individual losses as seen ...
user avatar
  • 1

1
2 3 4 5
16