Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
558
questions
0
votes
0answers
18 views
How to train LSTM model with multi-time series in timestamps
I have a problem, I don't know how to perform LSTM training in my case. I have flight data in the form of a time series, I have the complete route of the flight and what I need is to use this data to ...
0
votes
0answers
22 views
LSTM to generate squares and circles [closed]
I wanted long short-term memory (LSTM) to act as a static input receiver and generate a dynamic series of points as output. For example, I have set of squares and circles points as dataset and label (...
0
votes
1answer
28 views
Can vanishin gradient occur because of saturation cell state in LSTM without forget gate?
For very long input sequence, cell state can grow without bound and as a result output activation function (tanh) is saturated. In this case, can vanishing gradient occur in lstm without forget gate (...
0
votes
0answers
21 views
multiple time series but index with progression rate
I have a dataset with many object, each object is identified by an id, and many observations indexed by progression rate, like bellow
My objective is to find which algorithm use to ...
0
votes
0answers
14 views
How is it possible that validation MSE is low while test MSE is really high?
I'm having the following problem. I'm training a neural network LSTM using keras with the following architecture:
...
1
vote
0answers
21 views
Convergence time for LSTM and Vanilla feed-forward NN training/validation errors
While learning myself, I am doing a simple example of traffic forecasting with LSTM, comparing with vanilla feedforward NN (FFNN). I observed the following
When I have a large number of training ...
2
votes
0answers
12 views
Predict Sales as Counterfactual
Which modelling strategy (time frame, features, modelling technique) would you recommend to forecast 3-month sales for total customer base?
At my company, we often analyse the effect of e.g. ...
1
vote
0answers
34 views
When to use LSTM vs Lasso/Ridge Regression vs ARIMA?
I have a set of N time series and want to make predictions about the future values of these N elementary time signals.
From a first rough analysis, I can say that at a given moment in time, the N ...
0
votes
0answers
9 views
Keras MLP with time frame as input vs. RNN
I want to predict the discharge depending on the rain of the last x days of rain.
I would like to compare the performance of a standard MLP (multi layer perceptron) to a RNN. To give the MLP some ...
1
vote
0answers
17 views
Use LSTM to predict the proportion of steps with nonzero feature values
I am trying to do a simple regression for sequences. Each input $X_i$ is a $n=2000$ by 1 matrix, formatted as $n_i$ 0-s followed by $(n-n_i)$ 1-s. The output $y_i$ should be $n_i/n$, i.e. the ...
0
votes
0answers
9 views
Keras LSTM input_shape multivariat regression
I'm trying t set up a RNN to predict a discharge which depends on temperature and rain.
Of course there are lags time frame dependencies between the events, so discharge on day i depends on rain ...
1
vote
1answer
30 views
Is the number of cells in a keras LSTM or RNN layer equal to the number of time steps?
Say I have the following code to create a LSTM layer:
lstm_model = Sequential()
lstm_model.add(LSTM(128, batch_input_shape=(BATCH_SIZE, TIME_STEPS, FEATURES))
...
0
votes
0answers
15 views
Is my train loss, test loss and dev loss normal?
I have two data sets D1 and D2. Each data set is composed of two parts: 1)sentences and 2)scores. I want to learn scores for the sentences by an LSTM network. I have examined four scenarios:
...
0
votes
0answers
44 views
What should i do when my recurrent neural network doesn't improve? [duplicate]
I am training an LSTM network using Tensorflow 2, is there a way to debug it to see if its learning or to know what areas should be adjusted ?
Is there a way to debug to know if its the data, the ...
0
votes
0answers
11 views
How to pass this mock one-hot-encoded data through keras LSTM layer?
As (I think) I understand in Keras, LSTM layers expect input data to have 3-dimensions: (batch_size, timesteps, input_dim).
However, I'm really struggling to ...
1
vote
1answer
14 views
Neural network model that ouputs how similar a new input is to the previously trained ones for curiosty-based learning techniques
I don't find where to ask this or if this is too basic or complex to be solved, here is my question:
Is there a NN model like a CNN where you would recursively input images and get a single output ...
0
votes
0answers
33 views
Time Series Analysis for Coordinates using LSTM
I have a dataset in which ids of locations connected with corresponding latitudes and longtitudes. I want to predict next location of the activity after training my LSTM model. However, feeding the ...
0
votes
0answers
11 views
What is in LSTM memory cell
I understand the structure of LSTM, but I am unclear what is in the vector holding the LSTM memory, and I have not found any reference providing intuitive example on what exactly that vector is ...
0
votes
0answers
14 views
What is the functional expression for an LSTM and how is output scaled appropriately?
How do you express an LSTM as a function: first just a single computation, then their interactions together.
Furthermore which part of this expression is responsible for appropriate scaling (since ...
1
vote
1answer
19 views
Number of input neurons in a LSTM Autoencoder
My training data of shape (2110, 5, 29). When building a LSTM Autoencoder can the number of LSTM cells in my first LSTM layer be more than dimensions of the ...
0
votes
0answers
8 views
Computing custom gradient for LSTM equations
Consider an LSTM that takes in as input a sequence of N words $X_1,\cdots,X_N$. Each word is a vector $\in R^D$. The dimension of the LSTM neuron is $H$.
Suppose we are doing sentiment classification ...
4
votes
1answer
30 views
Can LSTMs classify multivariate time sequences?
I've been learning about LSTMs and I commonly see them applied to the same type of task. For example, given a 1D list of values, predict which class they belong to. Or, given 8 values from 8 sensors, ...
1
vote
0answers
19 views
Customer next visit behavior forecast
I'm currently working with retail data about a store and the goal is to predict when each customer will visit the store again e.g customer id = 1 will probably visit again in 6 days(recency)
My ...
0
votes
1answer
48 views
Difference between TimeDistributed and convLSTM2D layer in Keras? [closed]
I am working on RNN(CLSTM) and in examples i see somewhere
layers.convLSTM2D()
and somewhere i see
...
0
votes
0answers
13 views
Multilabel classification and regression on time-series data
I am trying to develop an LSTM network on a vehicle dataset I have obtained from my professor. The dataset in about vehicle driving in a roundabout. I have the following tasks to complete 1. Classify ...
7
votes
1answer
202 views
Is there a stronger Universal Approximation Theorem for LSTMs?
The Universal Approximation Theorem says that under certain conditions on your activation function, you can approximate any bounded continuous function with a feedforward neural network.
I believe ...
0
votes
0answers
9 views
Need clarification on Lstm neural network output and error
I think I'm well on my way to build a LSTM neural network from scratch, but in few details I still need some clarification:
If I was to use lstm to predict the next float value after a timeseries of ...
0
votes
1answer
58 views
Activation function between LSTM layers
I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
1
vote
1answer
24 views
What is the purpose of the update gate and how does it achieve it in a LSTM?
I understand how the forget gate works.
My understanding of the forget gate:
A sigmoid function is used to make each of the gate tensor's values $\Gamma_f^{<t>}$ range from 0 to 1.
The forget ...
0
votes
1answer
19 views
LSTM - Who are the inputs for those hidden cells?
I'm learning RNN and I'm understanding, but I have a specific question that I can not find answer:
What is the x input for the cells (pointed in yellow) for the ...
0
votes
1answer
43 views
LSTM and Time series: My statements are corrects about those concepts?
I'm having some problems to understand LSTM theory and how exactly works a time series data in a Deep learning. To see if my thoughts are correct, I will make some statements (explanation) about how I ...
0
votes
1answer
37 views
Train a RNN with unknown vocabulary size
I'm new to deep learning and i'm trying to code a Visual Question answering network.
I studied and (i think) understood how RNN and LSTM work.
From what i'he understood, i need to train my network ...
0
votes
0answers
15 views
How to apply CNN and LSTM together?
I have x_train shape as (7352, 128, 9) and y_train have 6 classes to classify. Can any one suggest how to apply CNN and LSTM together to these shapes ?
0
votes
0answers
18 views
Use historical data in Feed Forward networks to replace RNN / LSTM [duplicate]
I am very confused. My current understanding of RNN / LSTM is the following: All a they do is include the previous element of the sequence as an input. Along with other inputs from this itteration, it ...
0
votes
0answers
16 views
Why do test set predictions perform far better than a recursive forecast - time series forecast
I've been dealing with a LSTM stock forecaster, and I've been looking at articles like 1, 2. I know that the models are very likely overfitted, but nonetheless, the test set predictions are quite ...
0
votes
0answers
13 views
IMV-LSTM with one feature
I had the pleasure of reading this article on an Interpretable, Multi-Variate LSTM. It has a 2D array hidden state such that each row only interacts with one input variable, and input-to-hidden and ...
0
votes
0answers
15 views
How can LSTM predict correctly?
i made CNN-LSTM parallel layers to predict speed and steering values.
The layers look like (not the same values but similar structure) :
End-to-end Multi-Modal Multi-Task Vehicle Controlfor Self-...
0
votes
1answer
47 views
get predictability of word given sentence in python
I have a paragraph and I want to get the probability (p(word | context) ) of each word, given previous words, for various models (e.g. pre-trained LSTM).
Where can pretrained models would allow me to ...
0
votes
0answers
21 views
Accuracy metric in LSTM not considers time offset for multivariate time-series classification?
So this is a kind of complex question, so I hope I formulate it good enough.
I have a human activity detection task that binary classifies if a user does a specific action or not. For me, it is ...
1
vote
1answer
29 views
Do recurrent neural language models greedily model language probability?
Want to check my understanding of recurrent neural language models (in this case I'm working with a decoder in an encoder-decoder RNN but I don't think that matters significantly). I'm trying to ...
0
votes
0answers
20 views
Oversampling for imbalanced time series classification
I'm doing multivariate time series classification (two classes) with GRU/LSTM models. Each observation is a multivariate time series with one label (0 or 1). But the two classes are highly imbalanced. ...
1
vote
0answers
50 views
How to predict future sequence data? [closed]
I am currently doing a signal processing project. However, this is very different to projects I did before and I am struggling to find a good start point to do this.
My problem is as follows.
I ...
0
votes
0answers
50 views
How to update a keras LSTM weights to avoid Concept Drift
I´m trying to update a Keras LSTM to avoid the concept of drift. For that I´m following the approach proposed in this paper [1] on which they compute an anomaly score and they apply it to update the ...
0
votes
0answers
24 views
How should I map spectrogram data to another time series data using deep learning?
I have two sensors whose outputs I am trying to relate. Specifically, sensor A and sensor B I believe have a relationship.
Upon looking at their respective spectrograms, they clearly are related, ...
1
vote
1answer
35 views
What are number of hidden layers in LSTM?
I new to LSTM. I have not understood some terms used while implementing it in tensorflow. So I have ECG data, with each event having 60 heartbeat templates with each heartbeat template having 600 data ...
0
votes
0answers
57 views
Calculating EER with anomaly detection using LSTM in python
I have dataset features evaluated from the touch screen and built-in sensors on smartphones. I want to implement an anomaly detection code using LSTM autoencoder in python to compute EER value (Equal ...
1
vote
0answers
335 views
Mean absolute percentage error returning NAN in PyTorch [closed]
I'm using mean absolute percentage error (MAPE) as a loss function for an RNN, however during training I start getting NaN values. I first used MAPE to calculate error between sequences of 3D ...
1
vote
0answers
27 views
Stationarity in LSTM multivariate prediction
I am constructing a multivariate LSTM NN to predict a financial time series that is non-stationary at 1% significance level from an ADF test. However, the test rejects H0 of non-stationarity at 5%.
...
2
votes
1answer
20 views
RNN Regression outputting Same(ish) values
I have a sequence to sequence LSTM (encoder/decoder model) that I made following this tutorial. I'm trying to output a series of human poses (in the form of 3D coordinates) with shape (N, 17, 3). I'm ...
0
votes
1answer
66 views
Classification methods for univariate time series
Our team wants to develop a machine learning algorithm for classification of univariate data. Our data is a live feed from a position sensor placed in an injection molding machine. We want to be able ...
