Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
673
questions
0
votes
2answers
6 views
Is there a seq2seq model that can encode sentences that include numerical values?
I am trying to build a seq2seq model that encodes sentences which include numerical values. For example,
Patient's systolic blood pressure was 128.
Conventional ...
0
votes
0answers
7 views
How to decide the shape of the input of an LSTM layer with temporally combined data
I'm working on a time series with 8 features, where at the beginning of each day,
features 1 to 5 are unknown,
features 6 to 8 ...
0
votes
0answers
11 views
Forecasting with LSTM networks and use of final_hidden_states
I'm working on a seq2seq, stateless (return_state = False), forecasting problem.
Let's say I have 10 independent time series with dimensions (10,50,2) where 10 is the number of samples, 50 is the ...
1
vote
0answers
7 views
Larger batch size means faster training for RNN?
When training my LSTM, I realize time per eopch decreases as I increase the batch size. But this isn't true for training a ConvNet.
I wonder if this is because for RNNs with unspecified sequence ...
0
votes
0answers
13 views
How to shape input of LSTM
There are 200 factories. Each factory is represented as a 128 dim embedding in each quarter of one year. Thus, I have four data sheets for a whole year, with each sheet for each quarter for the 200 ...
0
votes
1answer
22 views
RNN functionality and outputs
This is a very basic question about the functionality of RNN / LSTM. My question is about the RNN outputs. Is there a single output for only the last element in the sequence, or is there an output for ...
0
votes
0answers
28 views
How should I handle sequential “event” data for time-series work?
The data I have is a sequence of varying "events", where there are a set number of different types of event, each with varying pieces of data associated with them (i.e. A B A A C, where A ...
2
votes
0answers
32 views
LSTM architecture for anomaly detection
I'm testing out different implementation of LSTM autoencoder on anomaly detection on 2D input.
My question is not about the code itself but about understanding the underlying behavior of each network.
...
1
vote
0answers
6 views
Can I skip the Keras Embedding Layer if I already transformed the data to Word2Vec (Google News 300 format)?
Trying to do sentiment analysis with an LSTM NN. I think I understand what the embedding layer does: map each word to a fixed-di-vector.
However, previously, for each text sample, I transformed each ...
1
vote
0answers
13 views
LSTM : Best approach for embedding phrases
I'm learning about LSTM for classifying an event sequence to determine if it required human intervention or if it's on its "normal" way What I have is a list of events with timestamps for ...
1
vote
0answers
14 views
LSTM Vanishing Gradients [closed]
I'm trying to implement the LSTM model for text classification where each sentence is about 1500 words. converted sentence to a sequence of values and fed to LSTM but gradients are becoming zero. I'm ...
2
votes
0answers
35 views
Canonical LSTM backpropagation equations
I'm trying to understand the underlying mechanisms of LSTM from a programming perspective. I am no math person, and a lot of articles and papers look like alphabet soup to me. But I thought that if I ...
0
votes
0answers
25 views
What to do if my LSTM model doesn't learn
I have taken text input then converted to a sequence of values and fed it to LSTM model where my loss is not reducing and accuracy is abnormal.
The above image is about training and validation ...
0
votes
0answers
20 views
Memory Capacity of Recurrent Neural Network
I am reading:
https://arxiv.org/abs/1610.06258
it states the following:
If I am understanding this correctly a vanilla RNN with 32 neurons and an output size of 5 would have a short term memory ...
0
votes
0answers
10 views
How to predict integer sequence from multi feature real input of time series with lstm seq2seq
I’ve been struggling with the seq2seq problem for a while. I have multi feature input of timeseries, about 10 features(less after PCA, but it is not in main focus of the question) and I need to ...
0
votes
0answers
19 views
Predicting several LTE cell's Traffic using a single LSTM Net
I'm new to StackExchange and I'm not sure how to address my issue.
I'm trying to predict LTE cell traffic using an LSTM model. I have more than 10k cells and I'd like to know how I can embed and feed ...
0
votes
0answers
57 views
Autoencoders For Multivariate Time-series Anomaly Detection
I have a multivariate time series of size (1e6, 15) and would like to fit a LSTM autoencoder. I prepare data with multivariate rolling windows (one step rolling) where each sample has (1, 5, 15) ...
0
votes
0answers
10 views
LSTM - unsure about input shape (dataset with multiple instances of patients)
I read through a lot of similar questions about the input of LSTM's, but since the applications were different from mine and mostly used one dimensional data it confused me more than it cleared ...
0
votes
1answer
68 views
Forecasting Sales LSTM - cannot capture peak values
I am trying to forecast retail sales for a company that have different stores. I currently use LSTM model which is designed as follow : data includes the info about sales between 2014-2020. After ...
5
votes
3answers
830 views
Forecasting Prices vs Returns by Deep Learning
The question: It seems that (univariate) forecasting stock market done by websites using DL and LSTM actually does not work that well if we focus on returns instead of prices. What is a relatively ...
1
vote
0answers
25 views
LSTM: Best way to handle categories, in practice
There is an issue emerging in the practical use of Long-Short-Term-Memory (LSTM) Deep Neural Nets (DNN) with my use case.
In typical machine learning scenarios one encounters in benchmark datasets, or ...
1
vote
0answers
38 views
Order of dropout and activation in 1D convolutional networks
I have a simple cnn-lstm network. There are two 1D convolutional layers after the input layer. Every 1D convolutional layer is followed by a dropout. What I observe is that when I have conv1D -> ...
0
votes
0answers
28 views
model.evaluate result differs from tf.keras.losses.MSE when using tf.data.Dataset
I am training the following RNN for univariate time series forecasting. Data are obtained from a pandas dataframe and converted into a tf.data.Dataset structure, ...
0
votes
0answers
20 views
Predicting with Stateful LSTM
Knowing the nature of my time series problem, I am using a stateful LSTM to forecast one step ahead.
My question is quite straightforward.
Do I need to explicitly save and pass the hidden cell in ...
0
votes
0answers
22 views
Neural network to generate barcodes
I am working on a problem regarding barcodes. This is my first time using sequence models practically. Summarizing it:
I have 681 32 character, case-insensitive, alphanumeric strings that are ...
1
vote
1answer
83 views
Does it make sense to use attention mechanism for seq-2-seq autoencoder for anomaly detection?
So I want to train LSTM sequence to sequence model, autoencoder, for anomaly detection. The idea is to train it on normal samples and when anomaly comes into model it will not be able to reconstruct ...
0
votes
1answer
14 views
conceptually, how to use NLP to predict a numeric output
suppose I'm trying to use medical notes to predict the cost of medical service. For example, a patient will call in, tell the operator how they feel, their diagnosis, etc etc, and the operator will ...
1
vote
1answer
41 views
Is there any direct relation in between accuracy and loss while performing text classification using neural network?
I am trying to perform text classification using the deep recurrent neural network. My network is incurring a huge loss of 94%, 80% and sometimes 100% with certain accuracy. It is surprising that with ...
0
votes
0answers
9 views
Can we model a linearly increasing time series with an lstm without making series stationary
If I want to model an linearly increasing times series using LSTM, should I remove the trend and seasonality from the time series before feeding it to LSTM or can LSTM efficiently model non stationary ...
0
votes
0answers
30 views
SARIMAX - LSTM Evaluation of forecast results
I am trying to build a sales forecasting algorithm for more than 2000 products and now sticking around ARIMA/SARIMAX and LSTM models. As I can see, both models are okay to use the data without make ...
0
votes
0answers
10 views
How to force rnn model to train with only peaks from input data?
I am using a recurrent neural network to model my traffic data. In the data, there is some peak and I would like to train my model for predicting future peaks. I mean forecasting the peak is much more ...
0
votes
0answers
36 views
Sales prediction with RNN or LSTM with multivariate time series - How to add features to prediction target but only predict product sales?
I try to predict future product sales with a RNN or a LSTM. I am using Multivariate Time Series.
I have a dataset with product sales of different shops. In my dataset, I have features about prices, ...
2
votes
0answers
23 views
Avoiding exploding gradients when forget gate is 1 and input/output gates are 0
Initially, the cell state equation was $C_t = C_{t-1} + i_t \odot \tanh(w_xx_t + w_hh_{t-1})$. Then to avoid exploding gradients, we added a forget gate such that $$C_t = f_t \odot C_{t-1} + i_t \odot ...
1
vote
1answer
30 views
Recurrent networks (RNN) and matrix definition of the input $x$ an weight matrix $W_{xh}$ for a time-series model?
I am playing with RNNs/LSTMs to do some time-series modelling. Now most of the tutorials on RNNs seem to focus on natural language processing tasks. Hence, most descriptions of RNNs begin with ...
1
vote
1answer
24 views
Can large # of epochs or smaller batchsize compensate for smaller data size in training lstms
I have about 40 time series (40 products) of weekly sales for 3 years ( = 156 data points for each series). So, in total I have about 6240 data points. To train a stateful or stateless lstm for ...
1
vote
0answers
29 views
Handling time series data with seasonality and trend for training LSTMs
Do I need to remove seasonality and trend from time series data, before using it for training lstm. Or, with sufficient data, can LSTMs recognize the pattern in trend and seasonality ? I have 3 years ...
0
votes
0answers
32 views
(LSTM with R) How Backtested cross validation RMSE can used for the actual LSTM forecasting model
This blog was super helpful to build LSTM code in R
https://www.r-bloggers.com/2018/04/time-series-deep-learning-forecasting-sunspots-with-keras-stateful-lstm-in-r/
So, I tried to use RMSE that is ...
1
vote
0answers
15 views
Can I frame the same time series forecasting problem either as stateful or stateless lstm
If I have to predict the weekly sales of the product from the past. My data is at weekly level for the product and has about 5 years of data i.e, 260 data points and have about 20 (independent) ...
2
votes
1answer
88 views
What does vanishing gradient problem exactly mean? [duplicate]
I'm currently doing some stuff with ML, and I created a LSTM model to recognize activities such as walking or running. I have read that LSTM has advantages over traditional RNNs due to addressing the ...
0
votes
0answers
22 views
Is masking needed for prediction in LSTM keras
I am trying to do sentence generator using 50D word embedding. If my training sentence is "hello my name is abc" here max words is 5. So my first training ...
1
vote
0answers
14 views
Dimension of Dataset Many to Many one to Many in LSTM keras
My understanding is In Many to Many dimension of if X is(batch_size,timestep,vector_size) and Y is ...
0
votes
0answers
9 views
Predict time series of parishable and seasonal product just with 1 year dataset
I want to predict the amount of demand for several types of fruit in a number of market with LSTM model.$ $
But I have a big problem, that I only have the dataset of one last year and because of that ...
0
votes
0answers
10 views
ML model type for making multi-step time-series predictions
Consider the following problem: making a prediction for 1 month based on 5 years of stock close prices.
What would be the best choices in terms of model structure for this problem? I have considered ...
0
votes
0answers
46 views
Using pretrained LSTM and Bert Models in CPU Only Environment - How to speed up Predictions?
I have trained two text classification models using GPU on Azure. The models are the following
Bert (ktrain)
Lstm Word2Vec (tensorflow)
Exaples of the code can be found here: NLP
I saved the models ...
3
votes
1answer
49 views
What is unrolling in LSTM
In keras LSTM if unroll set False does it mean that output of current timestep is equal to input next time step?
0
votes
0answers
75 views
How to normalize data for LSTM?
For learning purpose I am making simple dataset for LSTM, which can predict next number in sequence.
Here is my x value
...
1
vote
0answers
15 views
Using LSTM to predict next number [duplicate]
I trying to understand working of LSTM So I'm doing simple code to simulate its action.
...
0
votes
0answers
22 views
Can I use the predictions of an LSTM model as a feature for a gradient boosting regression model?
I have a transactional dataset of a retail company, for which I am trying to predict the sales on a monthly time interval, I have used an LSTM model with two features, the timestamp and the sales. I ...
0
votes
1answer
26 views
Do we need to truncate test dataset for seq2seq LSTM?
I am running a summarization model which uses a seq2seq biLSTM with an attention mechanism. It is a standard practice to truncate the input dataset during training to 400 - 500 tokens. My question is, ...
1
vote
1answer
28 views
Can we normalize both continuous and discrete numerical values
I have a sensor dataset with 16 features as numerical values (12 are continuous and 4 are discrete).
I am using LSTM model to fit the data and do some classification.
As both continuous and discrete ...