Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
687
questions
1
vote
0answers
13 views
LSTM Time Series Forecasting
I am working with Bitcoin block data and attempting to forecast target hashrate ~one month in advance. I have condensed the raw block-by-block data into daily data and calculating metrics like ...
0
votes
0answers
7 views
Can someone please explain the decoder attention step in the transformer framework?
I just wanted to clarify how the highlighted attention step in the decoder works.
From looking at the original paper (Vaswani, 2017 - "Attention is all you need"), I gather that the encoder ...
0
votes
0answers
6 views
Short Time Series forecasting for each individual unit
I have a dataset that contains department information for 100 units in a factory. for each unit I have 5 number which is the total number of contracts from 2016 to 2020 and I would like to predict ...
0
votes
0answers
15 views
Multi-step Multi-Feature RNNs [closed]
I'm currently looking into implementing two RNNs following the below architectures
where the target is a numerical value (the task is framed as a regression task) and the red boxes in the second ...
0
votes
0answers
12 views
Extract vector encoding from supervised LSTM
I want to use historical stock price data for some companies in the S&P 500 to predict future prices of the stocks and index as a whole. I'm taking the approach in this paper https://arxiv.org/pdf/...
0
votes
1answer
19 views
LSTM, regression or classification output layer
I am currently implementing an LSTM RNN with an underlying binary classification problem (0,1) and I am thinking what to choose as an output layer. From my point of view, both, regression and ...
0
votes
0answers
35 views
Term for categorization of neural networks for sequence modeling (RNN, LSTM, TCN..)
As written in the title, I wanted to ask if there is a common term to characterize neural networks like RNN, LSTM, TCNs etc.
Can I refer to them as sequential networks?
0
votes
0answers
9 views
LSTM time series prediction going out of phase
I have implemented an LSTM for a univariate time series prediction task. Objective is to predict the value 7 timestamp ahead. This is my result.
Observations:
When I predict for 1 timestamp ahead, ...
0
votes
0answers
25 views
Questions on LSTM RNN's
As the LSTM model is able to store information from previous timesteps in the cell state, I was wondering what (more specific: how many periods) to choose as input. My original plan was to use current ...
1
vote
1answer
16 views
Why shouldn't you mix variable size inputs in the same minibatch?
I am trying to build a CNN-LSTM architecure in tf.keras that classifies sequences of varying sizes. My training data is highly variable and I would have to crop/pad sequences in order to create ...
-1
votes
0answers
29 views
LSTM - stock prediction with stock prices + financial news
the problem is, I have financial news data (used sentiment analyser on it) and turned it into dataframe (neg,pos,neutre) and I have stock prices. both are in the form of time series.
I want to use ...
0
votes
0answers
9 views
Why is loss after training with normalized data higher than the loss of the same, but non-normalized, data?
I am toying around with training a simple LSTM model using time series data generated from the f(x) = sin(x) function to make predictions about the next value. I know this is non-sensical but I use it ...
0
votes
0answers
11 views
Applying the activation function in this way is equivalent to the other way - RNN
I don't understand why in RNN the 2 following ways of applying the activation functions are equivalent:
First way:
$$
h_t = W\sigma(h_{t-1}) + U x_t + b
$$
Second way:
$$
g_t = \sigma(Wg_{t-1} + U x_t ...
0
votes
0answers
18 views
Sliding window of LSTM
My input data has every 4 consecutive rows assigned for a different class. I want to use an LSTM for class prediction and for that I need to pass slice of 4 in the LSTM at once, I tried using the <...
0
votes
1answer
33 views
What are the inputs x for each timestep t?
I am currently having little understanding problems with seq2seq Models that are using LSTM. When I look at this image of a normal LSTM-Cell:
According to my information, these cells exist in the ...
2
votes
0answers
36 views
What is the Encoder in a seq2seq RNN doing?
I am learning about RNNs especially seq2seq Models that are using LSTM. I am wondering what exactly the Encoder in such models is doing. To ensure that I've understood the rest of the seq2seq-Model ...
2
votes
1answer
18 views
GRU Hidden State Output Formula Difference
Looking into GRU equation I see 2 type for final output. One is from, https://d2l.ai/chapter_recurrent-modern/gru.html#hidden-state, that is,
$
\mathbf{R}_t = \sigma(\mathbf{X}_t \mathbf{W}_{xr} + \...
1
vote
0answers
15 views
Vector vs. multi-step for LSTM time series forecasting [closed]
Assume we want to predict a sequence of values of a time series with an LSTM.
Question: What are the pros and cons of using a multi-step approach compared to merely using an output vector containing ...
1
vote
0answers
10 views
What is the difference between Grid LSTM and Multidimensional LSTM?
I don't see the difference between both architectures. Can you please explain me the difference between them?
0
votes
0answers
11 views
what's the output of RNN (LSTM) in the prediction of time series [duplicate]
It is a very basic question of RNN (LSTM). Here is the basic structure of RNN:
for a sample $(X^i,Y^i),$
input: $X^i = (x_1,\cdots,x_T);$ output: $Y^i = (y_1,\cdots,y_T).$
$$z_t = Uh_{t-1} + W x_t + b,...
1
vote
0answers
28 views
Is it possible to calculate AIC on LSTM?
I'm doing a forecast on returns of stocks using ARMA-GARCH models and LSTM. Because of the nature of the data, RMSE, MSE cannot be used. I instead found MAAPE.
Now what I'm trying to understand if I ...
0
votes
0answers
15 views
Using an LSTM for Multi-Time Series Data
I am currently working on a research project which aims to analyze the effects of pre-existing social and economic freedoms on the COVID-19 pandemic. In addition to more straightforward statistical ...
0
votes
0answers
26 views
Real time time series prediction with different input frequencies
I want to make predictions about an event occurring at the certain time every day. To be specific, I am thinking of one of the train X arriving at the station Y (e.g. at 3.30pm every day). My aim is ...
1
vote
1answer
24 views
What could cause my LSTM loss to decrease then increase?
I'm training 5 stacked Bi-LSTMs on an NLP task. The network fits well with a time series of length 30, and converges to around 0.97 AUROC. However, when I increase the length of the time series to ...
1
vote
0answers
17 views
Fill in the blank in sentences with bidirectional LSTM in Keras
I'm currently studying RNN, in particular LSTM and I was trying to figure out how to implement a bidirectional LSTM to fill in the missing word in a sentence. I have a doubt about the strucuture of ...
1
vote
0answers
11 views
Model for predicting medical outcomes on longitudinal data? Even with uneven time steps?
I have a dataset that originated from medical claims that has one subject/person on multiple rows, along with when the person visited the doctor, and all the diagnoses the person received during that ...
0
votes
0answers
22 views
Why does my accuracy change a lot when I use a dense layer in an LSTM?
I am using one to many LSTM approach, where,
I have an input layer, an LSTM layer, and a dense layer in Model 1.1
I have an input and an LSTM layer in my Model 2.2
You can visualize both the models ...
3
votes
2answers
128 views
Predicting millions of independent time series, using them to help each other
This is a very general problem faced by different types of companies. Predict future customer behavior over time.
Imagine that we have 1 million customers with their own resources over time, forming a ...
0
votes
2answers
29 views
Proper shape of LSTM dataset for keras
I understand that similar questions have been asked before, but they are all based on specific examples. I want to consider a very simple example: we have a sequence of 1000 numbers, and want an LSTM ...
0
votes
0answers
20 views
Neural networks: general questions about training and hidden layers in LSTM
Studying neural networks, I have some questions and I cannot find answers online, so here I am:
Since the candidate memory cells in LSTM are guaranteed by using the tanh function that the price range ...
1
vote
1answer
30 views
WHICH weight matrix are shared in RNN and which change upon Time?
In my opinion there are basically 4 weight matrix in the RNN. SO there are different names given in different scenarios but let me point out what I know about RNN in very simple terms. Please correct ...
0
votes
1answer
55 views
How to implement LSTM backpropagation through time?
I'm building a custom LSTM net based on this article. I got questions on how to implement the backpropagation, based on these formulas of the derivatives in an LSTM layer:
Question 1: The weights (w.....
1
vote
1answer
15 views
Ambiguity in definition of Long Term Forecasting
The well known definition of a long term forecasting model is based on the length of the forecast horizon. In case of long term, it is usually agreed to be in the order of years (ahead of the most ...
0
votes
0answers
13 views
Including a “year” effect in an LSTM sequence-to-sequence model for time-series forecasting
I am working on a problem to forecast some climate sensor data. I have data that includes daily sensor measurements from 1980 till present. Now I have written a Sequence-to-Sequence LSTM model that ...
0
votes
0answers
77 views
RNN vs ResNet for multivariate time series prediction
All others being equal, would a ResNet-based or RNN-based neural network (with/without an attention mechanism) perform better for forecasting a multivariate time series?
Related:
Deep learning for ...
1
vote
2answers
17 views
Is there a seq2seq model that can encode sentences that include numerical values?
I am trying to build a seq2seq model that encodes sentences which include numerical values. For example,
Patient's systolic blood pressure was 128.
Conventional ...
0
votes
0answers
8 views
How to decide the shape of the input of an LSTM layer with temporally combined data
I'm working on a time series with 8 features, where at the beginning of each day,
features 1 to 5 are unknown,
features 6 to 8 ...
0
votes
0answers
13 views
Forecasting with LSTM networks and use of final_hidden_states
I'm working on a seq2seq, stateless (return_state = False), forecasting problem.
Let's say I have 10 independent time series with dimensions (10,50,2) where 10 is the number of samples, 50 is the ...
1
vote
0answers
9 views
Larger batch size means faster training for RNN?
When training my LSTM, I realize time per eopch decreases as I increase the batch size. But this isn't true for training a ConvNet.
I wonder if this is because for RNNs with unspecified sequence ...
0
votes
0answers
18 views
How to shape input of LSTM
There are 200 factories. Each factory is represented as a 128 dim embedding in each quarter of one year. Thus, I have four data sheets for a whole year, with each sheet for each quarter for the 200 ...
0
votes
1answer
26 views
RNN functionality and outputs
This is a very basic question about the functionality of RNN / LSTM. My question is about the RNN outputs. Is there a single output for only the last element in the sequence, or is there an output for ...
0
votes
0answers
28 views
How should I handle sequential “event” data for time-series work?
The data I have is a sequence of varying "events", where there are a set number of different types of event, each with varying pieces of data associated with them (i.e. A B A A C, where A ...
2
votes
0answers
33 views
LSTM architecture for anomaly detection
I'm testing out different implementation of LSTM autoencoder on anomaly detection on 2D input.
My question is not about the code itself but about understanding the underlying behavior of each network.
...
1
vote
0answers
10 views
Can I skip the Keras Embedding Layer if I already transformed the data to Word2Vec (Google News 300 format)?
Trying to do sentiment analysis with an LSTM NN. I think I understand what the embedding layer does: map each word to a fixed-di-vector.
However, previously, for each text sample, I transformed each ...
1
vote
0answers
15 views
LSTM : Best approach for embedding phrases
I'm learning about LSTM for classifying an event sequence to determine if it required human intervention or if it's on its "normal" way What I have is a list of events with timestamps for ...
1
vote
0answers
16 views
LSTM Vanishing Gradients [closed]
I'm trying to implement the LSTM model for text classification where each sentence is about 1500 words. converted sentence to a sequence of values and fed to LSTM but gradients are becoming zero. I'm ...
2
votes
0answers
41 views
Canonical LSTM backpropagation equations
I'm trying to understand the underlying mechanisms of LSTM from a programming perspective. I am no math person, and a lot of articles and papers look like alphabet soup to me. But I thought that if I ...
0
votes
0answers
36 views
What to do if my LSTM model doesn't learn
I have taken text input then converted to a sequence of values and fed it to LSTM model where my loss is not reducing and accuracy is abnormal.
The above image is about training and validation ...
0
votes
0answers
32 views
Memory Capacity of Recurrent Neural Network
I am reading:
https://arxiv.org/abs/1610.06258
it states the following:
If I am understanding this correctly a vanilla RNN with 32 neurons and an output size of 5 would have a short term memory ...
0
votes
0answers
13 views
How to predict integer sequence from multi feature real input of time series with lstm seq2seq
I’ve been struggling with the seq2seq problem for a while. I have multi feature input of timeseries, about 10 features(less after PCA, but it is not in main focus of the question) and I need to ...
