Questions tagged [lstm]

A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

Existe-t-il une relation entre la taille du window size et batch size dans LSTM? [closed]

J'utilise le deep learning sur un petit ensemble de données (2000 lignes en gros avec 20 features). Je voudrai savoir s'il existait une relation entre le choix de la taille du window size et la taille ...
0 votes
0 answers
9 views

LSTM/GRU and K-FoldCV

I am applying LSTM and GRU models to a financial problem where the dataset is a time series composed of 365 rows (365 continuous days) and 23 columns (my outcome is the closing price of a financial ...
  • 121
0 votes
0 answers
29 views

What inference can be drawn from graph and underlying model

I am using lstm for time series prediction, here is graph of loss,accuracy, val_loss, accuracy, Model is able to achieve following 1)Can someone share comments/observations/insights on graph and ...
3 votes
2 answers
214 views

What exactly is meant by isotropic and anisotropic with word vectors

From this paper https://aclanthology.org/D19-1006.pdf "How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings" When they say word ...
  • 112
0 votes
1 answer
18 views

Fitting an autoregressive models not using the classical Maximum Likelihood Estimator?

I know for example, one could try the Recurrent Neural Networks approaches. But are there any other options to these? I don't know if the questions is rather absurd. I am thinking of models like ...
2 votes
0 answers
17 views

Can recurrent dropout be used as a Bayesian approximation?

To express model uncertainty in LSTM, we can use dropout as a Bayesian approximation for Gaussian model according to this, (dropout is kept during inference and that will result in a distribution of ...
0 votes
0 answers
15 views

RNN: How to feed multiple different time series to the network?

I'm new to RNN's and asking for advice on how to feed data to an RNN such as a LSTM. I'm trying to forecast wind speeds at one location. For this location i have historical values as a time series. ...
  • 23
0 votes
0 answers
18 views

Understanding train vs validation loss chart

I am training an LSTM to a univariate time series and I have some questions about how to evaluate the train vs validations loss charts and which number of epochs to use in the model. To give more ...
  • 339
0 votes
0 answers
16 views

How to deal with multivariate time series forecasting with neural networks, considering known values in the future for some of them?

I would like to deal with a time series forecasting problem where we have a multivariate time series, involving 5 different time series (4 inputs and 1 output). That is, at each time point we have 5 ...
  • 566
1 vote
0 answers
18 views

How can I call a point in the future of a time series that I would like to forecast?

I'm a beginner in time series forecasting and I'm not sure if I understood correctly the meaning of forecast horizon. I was considering horizons as points in the future for which we would like to ...
  • 566
0 votes
0 answers
11 views

Convolutional-LSTM model predicts same image every timestep

So i am training a convolutional-lstm model which uses VGG19 as the encoder model, which encodes a sequence of images and passes this sequences to a lstm which is then fed into a fully connected layer ...
  • 1
0 votes
1 answer
50 views

Understanding the loss funcion in DeepAR

The loss function looks like below, where N is the number of time series. ignore N for now and say N = 1. T is length of prediction horizon. During training $t_0$ starts from encoder time step 0 all ...
1 vote
0 answers
54 views

Combining multiple stock time series to one data set for LSTM

I am trying to predict daily stock return volatility using an LSTM network. My data comprises price data of five different stocks, over the same time frame. My question, to which I have not found an ...
  • 11
0 votes
0 answers
17 views

Should I Use rolling average for time series forecasting in this situation? (VAR, XGBoost regression, LSTM)

I am trying to perform multivariate time series forecasting using VAR, XGBoost regressor, and LSTM for comparison and I'm aware that it is common practice to smooth out the data by taking a rolling ...
  • 1
0 votes
0 answers
21 views

understanding what pattern DeepAR learns and how it works

I am trying to understand how the following simple pattern from a synthetic count time series is learnt using DeepAR. https://arxiv.org/abs/1704.04110 The count time series is generated from a ...
1 vote
0 answers
8 views

Peaks in error during training - Regression problem with DL model (LSTM)

During training I get unusual behavior of my model. Peaks show up in the both validation and training errors that I could not understand. I use MSE as a loss 1 and similar behavior appears in other ...
  • 11
0 votes
0 answers
16 views

Compare univariate ARIMA and LSTM predictions

I am trying to learn how to make predictions for a univariate time series using an LSTM neural network. I know how to work with time series using traditional ARIMA models but with the LSTM I am ...
  • 339
1 vote
0 answers
52 views

What does it mean forecast horizon in time series forecasting?

I'm a beginner in time series forecasting and I'm not sure if I understood correctly the meaning of forecast horizon. I was considering horizons as points in the future for which we would like to ...
  • 566
0 votes
1 answer
31 views

Learning a stochastic pattern in a count TS using DeepAR [duplicate]

I am trying to the learn the following pattern of count time series of vehicle demand every hour. The count time series is generated from a negative binomial distribution with parameters n = 9 and p =...
0 votes
0 answers
19 views

Stacked RNN to overcome vanishing gradient

Let me clarify that with vanishing gradient i don't mean the gradient to become zero (like for DNN with multiple sigmoidal layers) but learning long term dependencies.. So, LSTM are well known that ...
0 votes
0 answers
21 views

How to model token sequence along with numerical sequence?

My data contains sequences. Here are two example sequences and their labels: ...
  • 867
0 votes
0 answers
11 views

What is the procedure for data preprocessing for time-dependent LSTM classifier?

I attempt a beginner level LSTM classification task with a time-series numerical data, but my task is finding changes in features over time (in which those changes describe the outcome or the classes),...
1 vote
1 answer
12 views

How to weigh time-series data with a utility function with jumps?

I have data measured at fixed intervals in one process. It needs to be weighed with cost value that is generated in another process that is analogue and the costs it generates have jumps. Imagine a ...
0 votes
0 answers
28 views

Predict certain values in time series (not next time step)

I have input of numbers as -5,-4..4,5 I only want to predict which value will appear next first 5 or -5, sometime in the furure, not necessarily in the next step How should I define my model?
0 votes
0 answers
76 views

High validation accuracy and training accuracy but low test accuracy

I have a LSTM model that has good training accuracy(~90%) and excellent validation accuracy(> 95%) but it gives poor results when I test it on data it hasn't seen. I am training hyperparameters ...
1 vote
2 answers
18 views

What is the difference in ANN with lagged observation as predictors and LSTM?

I am trying to understand how ANN is different from LSTM when we include lagged variables as predictors. Is the difference solely in forget gate for LSTM?
  • 11
1 vote
1 answer
18 views

Optimal way to create a feature set?

I have a time series data (say weather data for each day for one week) that changes at each time step. Along with this, I have some data that is fixed (eg - the latitude and longitude of this place ...
1 vote
1 answer
30 views

Turning point of Adam optimizer for LSTM

I'm training a two-layer LSTM with Adam optimizer for time-series data. I encountered several times that there was a "turning point" of the MAE vs. epochs plot. Is this a normal behavior?
1 vote
1 answer
71 views

Sequential Binary Imbalanced data classification with LSTM

I'm building an LSTM sequential Binary Classification Model, the data is highly imbalanced like say Fraud detection case. After building an LSTM model on Sequential Vectorised data, I'm getting a very ...
0 votes
0 answers
6 views

How trained to LSTM models need to be to compare them?

If I want to compare various LSTM models on the same training set, either by size (number and/or size of layers) or by hyperparameter tuning, how trained do they need to be. If 100 epochs was ...
0 votes
0 answers
20 views

Tries to replicate a NN and does not learn

I am trying to replicate this model from matlab but for some reason, my loss function does not decrease a lot and stays near 1.80 (initial value) - model Here is my code of the model and the training-...
  • 1
0 votes
0 answers
31 views

How can I normalize extracted latent space from Autoencoder?

I'm currently training two LSTM-autoencoder for each simulation data and real data. The data are vectors and I wanna match the latent space values of each encoder. Although it converges well, but I ...
0 votes
0 answers
19 views

GoogleNet-LSTM, cross entropy loss does not decrease [duplicate]

...
  • 1
0 votes
1 answer
83 views

How to invert differencing in time series data if I am making multiple steps prediction?

I have a time-series that I would like to use for predicting 36 timesteps in advance using LSTM. It is not stationary so I differenced the series by subtracting each point from the next one. My ...
1 vote
0 answers
465 views

CNN-LSTM or LSTM better for univariate Time series forecasting?

I am working with simulated univariate sequential data and the goal is to forecast that data. I was wondering which model CNN-LSTM or LSTM is better for predicting univariate time series data. Both ...
  • 121
0 votes
0 answers
14 views

Training loss first fall then rise when training a bidirectional LSTM model for a NER task

I'm training a bidirectional-LSTM (Bi-LSTM) + conditional random field (CRF) model for a named entity recognition task. The training set contains 7033 labeled sentences, and the validation set for ...
0 votes
1 answer
332 views

Interconnections between embeddings layer and LSTM layer

I'm trying to build a text classifier with keras using word embeddings (glove) and a RNN (in this case a LSTM) using keras. I searched in several sites and decided to start with this configuration: <...
1 vote
0 answers
219 views

Where to add Dropout in CNN-LSTM?

I am creating a CNN-LSTM model to forecast sequential simulation data. At the moment I am not sure what the best place is to use Dropout in a CNN-LSTM architecture. Is it between the CNN and LSTM ...
  • 121
0 votes
1 answer
179 views

How to minimize prediction lag using LSTM model?

I am trying to use LSTM model to do prediction on index, but I find that it has quite obvious time lag on prediction. I tried ...
  • 115
1 vote
1 answer
50 views

How to forecast a Macro trend by multiple Index time-series Using LSTM Model?

I am new in machine learning and I found that lots of article only train the LSTM model by only one stock and do the forecast. ...
  • 115
0 votes
0 answers
26 views

How to train LSTM model with multiple index futures to forecast marco economy trend? [duplicate]

I want to try predicting the macro economy trend by grouping different index futures. I have came up 2 approaches and listed below. May I ask what is the right approach for handling multiple ...
  • 115
0 votes
0 answers
21 views

LSTM architecture for forecasting amplitude varying univariate-timeseries

Hi I am wondering if someone knows what a good LSTM architecture for amplitude varying univariate time series. I used a very simple LSTM network (1 lstm layer with 100 neuron's and FCC layer with a ...
  • 121
0 votes
0 answers
39 views

is it good to have 100% accuracy on validation?

i'm still new in machine learning. currently i'm creating an anomaly detection for flight data. it is a multivariate time series data that include timestamp, latitude, longitude, velocity and altitude ...
0 votes
0 answers
35 views

Understanding types of LSTM and their use cases

I'm currently considering to use RNN/LSTM for a predictive modelling project that involves time-variant points. From looking at the following types of LSTM/RNN (in the picture below), I want to try ...
0 votes
0 answers
24 views

Similar results between feedforward neural networks, recurrent neural network and LSTM for time series data - Is this standard?

Tl;dr: I have trained feedforward neural networks, recurrent neural networks and LSTM networks to predict behaviour of weather temperature. The results are almost all the same (see below). Is this ...
  • 133
0 votes
0 answers
34 views

Question about understanding Weights of Keras LSTM model

I am implementing Federated Learning (FL) using Keras LSTM. Starting with the simple example where multiple models are trained at different clients. Each client shares their model weights with the ...
  • 444
0 votes
0 answers
333 views

Evaluation for LSTM model

I have created a model for text generation using LSTM. I am having chess sequences learned, reporting only the pieces moved during the moves. So when I move a pawn on my game there will be "p&...
0 votes
1 answer
44 views

Baseline model for predicting the load forecast

I have a model which uses the historical data to predict the electricity load consumption. I want to compare my model with a baseline model to show the performance, however I can not find a good base ...
  • 23
0 votes
0 answers
29 views

How to increase accuracy and to receive better forecasts?

I am building a LSTM network and I am using the following structure: ...
0 votes
0 answers
122 views

The correct pre-processing of time-series data when using LSTM

I have a time-series data that I would like to use for forecasting the data trend using LSTM. I followed https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-...

1
2 3 4 5
16