Cross Validated
Sponsored by
Sponsored logo

Questions tagged [lstm]

Ask Question

A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.

673 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
2answers
6 views

Is there a seq2seq model that can encode sentences that include numerical values?

I am trying to build a seq2seq model that encodes sentences which include numerical values. For example, Patient's systolic blood pressure was 128. Conventional ...
0
votes
0answers
7 views

How to decide the shape of the input of an LSTM layer with temporally combined data

I'm working on a time series with 8 features, where at the beginning of each day, features 1 to 5 are unknown, features 6 to 8 ...
0
votes
0answers
11 views

Forecasting with LSTM networks and use of final_hidden_states

I'm working on a seq2seq, stateless (return_state = False), forecasting problem. Let's say I have 10 independent time series with dimensions (10,50,2) where 10 is the number of samples, 50 is the ...
1
vote
0answers
7 views

Larger batch size means faster training for RNN?

When training my LSTM, I realize time per eopch decreases as I increase the batch size. But this isn't true for training a ConvNet. I wonder if this is because for RNNs with unspecified sequence ...
0
votes
0answers
13 views

How to shape input of LSTM

There are 200 factories. Each factory is represented as a 128 dim embedding in each quarter of one year. Thus, I have four data sheets for a whole year, with each sheet for each quarter for the 200 ...
0
votes
1answer
22 views

RNN functionality and outputs

This is a very basic question about the functionality of RNN / LSTM. My question is about the RNN outputs. Is there a single output for only the last element in the sequence, or is there an output for ...
0
votes
0answers
28 views

How should I handle sequential “event” data for time-series work?

The data I have is a sequence of varying "events", where there are a set number of different types of event, each with varying pieces of data associated with them (i.e. A B A A C, where A ...
2
votes
0answers
32 views

LSTM architecture for anomaly detection

I'm testing out different implementation of LSTM autoencoder on anomaly detection on 2D input. My question is not about the code itself but about understanding the underlying behavior of each network. ...
1
vote
0answers
6 views

Can I skip the Keras Embedding Layer if I already transformed the data to Word2Vec (Google News 300 format)?

Trying to do sentiment analysis with an LSTM NN. I think I understand what the embedding layer does: map each word to a fixed-di-vector. However, previously, for each text sample, I transformed each ...
1
vote
0answers
13 views

LSTM : Best approach for embedding phrases

I'm learning about LSTM for classifying an event sequence to determine if it required human intervention or if it's on its "normal" way What I have is a list of events with timestamps for ...
1
vote
0answers
14 views

LSTM Vanishing Gradients [closed]

I'm trying to implement the LSTM model for text classification where each sentence is about 1500 words. converted sentence to a sequence of values and fed to LSTM but gradients are becoming zero. I'm ...
2
votes
0answers
35 views

Canonical LSTM backpropagation equations

I'm trying to understand the underlying mechanisms of LSTM from a programming perspective. I am no math person, and a lot of articles and papers look like alphabet soup to me. But I thought that if I ...
0
votes
0answers
25 views

What to do if my LSTM model doesn't learn

I have taken text input then converted to a sequence of values and fed it to LSTM model where my loss is not reducing and accuracy is abnormal. The above image is about training and validation ...
0
votes
0answers
20 views

Memory Capacity of Recurrent Neural Network

I am reading: https://arxiv.org/abs/1610.06258 it states the following: If I am understanding this correctly a vanilla RNN with 32 neurons and an output size of 5 would have a short term memory ...
0
votes
0answers
10 views

How to predict integer sequence from multi feature real input of time series with lstm seq2seq

I’ve been struggling with the seq2seq problem for a while. I have multi feature input of timeseries, about 10 features(less after PCA, but it is not in main focus of the question) and I need to ...
0
votes
0answers
19 views

Predicting several LTE cell's Traffic using a single LSTM Net

I'm new to StackExchange and I'm not sure how to address my issue. I'm trying to predict LTE cell traffic using an LSTM model. I have more than 10k cells and I'd like to know how I can embed and feed ...
0
votes
0answers
57 views

Autoencoders For Multivariate Time-series Anomaly Detection

I have a multivariate time series of size (1e6, 15) and would like to fit a LSTM autoencoder. I prepare data with multivariate rolling windows (one step rolling) where each sample has (1, 5, 15) ...
0
votes
0answers
10 views

LSTM - unsure about input shape (dataset with multiple instances of patients)

I read through a lot of similar questions about the input of LSTM's, but since the applications were different from mine and mostly used one dimensional data it confused me more than it cleared ...
0
votes
1answer
68 views

Forecasting Sales LSTM - cannot capture peak values

I am trying to forecast retail sales for a company that have different stores. I currently use LSTM model which is designed as follow : data includes the info about sales between 2014-2020. After ...
5
votes
3answers
830 views

Forecasting Prices vs Returns by Deep Learning

The question: It seems that (univariate) forecasting stock market done by websites using DL and LSTM actually does not work that well if we focus on returns instead of prices. What is a relatively ...
1
vote
0answers
25 views

LSTM: Best way to handle categories, in practice

There is an issue emerging in the practical use of Long-Short-Term-Memory (LSTM) Deep Neural Nets (DNN) with my use case. In typical machine learning scenarios one encounters in benchmark datasets, or ...
1
vote
0answers
38 views

Order of dropout and activation in 1D convolutional networks

I have a simple cnn-lstm network. There are two 1D convolutional layers after the input layer. Every 1D convolutional layer is followed by a dropout. What I observe is that when I have conv1D -> ...
0
votes
0answers
28 views

model.evaluate result differs from tf.keras.losses.MSE when using tf.data.Dataset

I am training the following RNN for univariate time series forecasting. Data are obtained from a pandas dataframe and converted into a tf.data.Dataset structure, ...
0
votes
0answers
20 views

Predicting with Stateful LSTM

Knowing the nature of my time series problem, I am using a stateful LSTM to forecast one step ahead. My question is quite straightforward. Do I need to explicitly save and pass the hidden cell in ...
0
votes
0answers
22 views

Neural network to generate barcodes

I am working on a problem regarding barcodes. This is my first time using sequence models practically. Summarizing it: I have 681 32 character, case-insensitive, alphanumeric strings that are ...
1
vote
1answer
83 views

Does it make sense to use attention mechanism for seq-2-seq autoencoder for anomaly detection?

So I want to train LSTM sequence to sequence model, autoencoder, for anomaly detection. The idea is to train it on normal samples and when anomaly comes into model it will not be able to reconstruct ...
0
votes
1answer
14 views

conceptually, how to use NLP to predict a numeric output

suppose I'm trying to use medical notes to predict the cost of medical service. For example, a patient will call in, tell the operator how they feel, their diagnosis, etc etc, and the operator will ...
1
vote
1answer
41 views

Is there any direct relation in between accuracy and loss while performing text classification using neural network?

I am trying to perform text classification using the deep recurrent neural network. My network is incurring a huge loss of 94%, 80% and sometimes 100% with certain accuracy. It is surprising that with ...
0
votes
0answers
9 views

Can we model a linearly increasing time series with an lstm without making series stationary

If I want to model an linearly increasing times series using LSTM, should I remove the trend and seasonality from the time series before feeding it to LSTM or can LSTM efficiently model non stationary ...
0
votes
0answers
30 views

SARIMAX - LSTM Evaluation of forecast results

I am trying to build a sales forecasting algorithm for more than 2000 products and now sticking around ARIMA/SARIMAX and LSTM models. As I can see, both models are okay to use the data without make ...
0
votes
0answers
10 views

How to force rnn model to train with only peaks from input data?

I am using a recurrent neural network to model my traffic data. In the data, there is some peak and I would like to train my model for predicting future peaks. I mean forecasting the peak is much more ...
0
votes
0answers
36 views

Sales prediction with RNN or LSTM with multivariate time series - How to add features to prediction target but only predict product sales?

I try to predict future product sales with a RNN or a LSTM. I am using Multivariate Time Series. I have a dataset with product sales of different shops. In my dataset, I have features about prices, ...
2
votes
0answers
23 views

Avoiding exploding gradients when forget gate is 1 and input/output gates are 0

Initially, the cell state equation was $C_t = C_{t-1} + i_t \odot \tanh(w_xx_t + w_hh_{t-1})$. Then to avoid exploding gradients, we added a forget gate such that $$C_t = f_t \odot C_{t-1} + i_t \odot ...
1
vote
1answer
30 views

Recurrent networks (RNN) and matrix definition of the input $x$ an weight matrix $W_{xh}$ for a time-series model?

I am playing with RNNs/LSTMs to do some time-series modelling. Now most of the tutorials on RNNs seem to focus on natural language processing tasks. Hence, most descriptions of RNNs begin with ...
1
vote
1answer
24 views

Can large # of epochs or smaller batchsize compensate for smaller data size in training lstms

I have about 40 time series (40 products) of weekly sales for 3 years ( = 156 data points for each series). So, in total I have about 6240 data points. To train a stateful or stateless lstm for ...
1
vote
0answers
29 views

Handling time series data with seasonality and trend for training LSTMs

Do I need to remove seasonality and trend from time series data, before using it for training lstm. Or, with sufficient data, can LSTMs recognize the pattern in trend and seasonality ? I have 3 years ...
0
votes
0answers
32 views

(LSTM with R) How Backtested cross validation RMSE can used for the actual LSTM forecasting model

This blog was super helpful to build LSTM code in R https://www.r-bloggers.com/2018/04/time-series-deep-learning-forecasting-sunspots-with-keras-stateful-lstm-in-r/ So, I tried to use RMSE that is ...
1
vote
0answers
15 views

Can I frame the same time series forecasting problem either as stateful or stateless lstm

If I have to predict the weekly sales of the product from the past. My data is at weekly level for the product and has about 5 years of data i.e, 260 data points and have about 20 (independent) ...
2
votes
1answer
88 views

What does vanishing gradient problem exactly mean? [duplicate]

I'm currently doing some stuff with ML, and I created a LSTM model to recognize activities such as walking or running. I have read that LSTM has advantages over traditional RNNs due to addressing the ...
0
votes
0answers
22 views

Is masking needed for prediction in LSTM keras

I am trying to do sentence generator using 50D word embedding. If my training sentence is "hello my name is abc" here max words is 5. So my first training ...
1
vote
0answers
14 views

Dimension of Dataset Many to Many one to Many in LSTM keras

My understanding is In Many to Many dimension of if X is(batch_size,timestep,vector_size) and Y is ...
0
votes
0answers
9 views

Predict time series of parishable and seasonal product just with 1 year dataset

I want to predict the amount of demand for several types of fruit in a number of market with LSTM model.$ $ But I have a big problem, that I only have the dataset of one last year and because of that ...
0
votes
0answers
10 views

ML model type for making multi-step time-series predictions

Consider the following problem: making a prediction for 1 month based on 5 years of stock close prices. What would be the best choices in terms of model structure for this problem? I have considered ...
0
votes
0answers
46 views

Using pretrained LSTM and Bert Models in CPU Only Environment - How to speed up Predictions?

I have trained two text classification models using GPU on Azure. The models are the following Bert (ktrain) Lstm Word2Vec (tensorflow) Exaples of the code can be found here: NLP I saved the models ...
3
votes
1answer
49 views

What is unrolling in LSTM

In keras LSTM if unroll set False does it mean that output of current timestep is equal to input next time step?
0
votes
0answers
75 views

How to normalize data for LSTM?

For learning purpose I am making simple dataset for LSTM, which can predict next number in sequence. Here is my x value ...
1
vote
0answers
15 views

Using LSTM to predict next number [duplicate]

I trying to understand working of LSTM So I'm doing simple code to simulate its action. ...
0
votes
0answers
22 views

Can I use the predictions of an LSTM model as a feature for a gradient boosting regression model?

I have a transactional dataset of a retail company, for which I am trying to predict the sales on a monthly time interval, I have used an LSTM model with two features, the timestamp and the sales. I ...
0
votes
1answer
26 views

Do we need to truncate test dataset for seq2seq LSTM?

I am running a summarization model which uses a seq2seq biLSTM with an attention mechanism. It is a standard practice to truncate the input dataset during training to 400 - 500 tokens. My question is, ...
1
vote
1answer
28 views

Can we normalize both continuous and discrete numerical values

I have a sensor dataset with 16 features as numerical values (12 are continuous and 4 are discrete). I am using LSTM model to fit the data and do some classification. As both continuous and discrete ...

1
2 3 4 5
…
14