Questions tagged [lstm]

A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Linear Returns vs Log Returns to make a Forex time series stationary

I have a GBPUSD Forex time series. I am preparing the series as an input to LSTM. As a best practice for LSTM I am making the series stationary and I have tried both linear returns and log returns for ...
0
votes
0answers
7 views

Issues of constructing multiple encoders in seq2seq network [closed]

I'm trying to construct a seq2seq network for time series prediction, the encoder contains multiple encoders with different inputs. All inputs are with 3d shape (samples, timesteps, features). As the ...
0
votes
1answer
13 views

Generating text using LSTM given condition vector

I know that you can use an RNN to generate text given the first few letters ...
1
vote
0answers
22 views

Visual attention on images in LSTM neural network - how to implement?

I have developed a bi-directional LSTM on images, and I want to get some kind of visual feedback on how the neural network is working, something that I think is called visual attention. I want to ...
0
votes
1answer
17 views

What would be the dimensions of the weights in an LSTM cell? [closed]

How do I know the dimensions of the weights mentioned in this diagram? Sorry if this is a dumb question, I'm pretty new to LSTM's and I'm trying to code one from scratch. Also, is it better to stack ...
0
votes
1answer
21 views

How to model properly sequential data when the output has to be used as part of the next input? Model off completely when it makes single mistake

I have time series data and am fitting a (LSTM) neural network. The time series data include let's say a brain wave (var1) as well as the previous state (prev_state) and I want to predict a state (...
0
votes
0answers
4 views

LSTM model convergence to a fixed state

I understand that this question has been asked previously https://stats.stackexchange.com/questions/253967/why-would-an-lstm-converge-to-a-fixed-state-when-generating-sequences?r=SearchResults&s=1|...
1
vote
0answers
12 views

LSTM for multivariate time-series classification with unequal timesteps

I am attempting to use RNN or LSTM for multivariate time-series classification of my data. A sample corresponds to an actor, a time-step corresponds to an action and a single action consists of many ...
0
votes
0answers
17 views

Is Predictions using machine learning algorithm like LSTM, Linear Regression, Transformers,etc. really a prediction?

When I came through the models like Linear Regression , LSTM , Transformer model a doubt was raised . we train a model using train values, ex:" model.fit(xtrain,ytrain, etc)", and in the ...
1
vote
0answers
48 views

LSTM vector sum of inputs as memory

I want to set up a long short-term memory (LSTM) network to have the vector sum of the inputs, which live in R^d, as its memory (ct). what is the required choices of the weight, and activation ...
0
votes
0answers
9 views

Univariate time-series prediction using LSTM

I am trying to implement the vanilla LSTM prediction model from scratch, but I am kinda stuck on getting output from the hidden state (ht -> yt). These are what I am doing right now: The plot of ...
0
votes
0answers
17 views

Using an RNN to generate a sequence with a static end point using TF and Keras

I'm currently learning about how to use neural networks while working on a project of mine. In the project I'm attempting to have a neural network create a path from a starting point (0,0) to an end ...
0
votes
0answers
8 views

Sequence sub matching for a big database

I have database (~1M) of 1D sequence (variable lengths). I want to be able to match a query sequence against the DB in real time. The query may be a sub sequence of one (or multiple) of the database ...
0
votes
0answers
26 views

Trying to increase the F1 score for an NLP data extraction problem

We have a problem reaching a decent F1 score when tackling this NLP data extraction problem. When given a group (i.e. Female dogs, male dogs) we want to extract relevant numerical data from a ...
2
votes
0answers
22 views

Application of Wavelet Transform and Differencing on Time Series Data (to denoise and remove seasonal adjustment and other trends)

I am working on an LSTM model to predict time series data (stock prices) and I would like an opinion whether to denoise my data or not before feeding it into the model. According to INVESTOPEDIA, ...
0
votes
0answers
17 views

Predicting sequence element based on the previous M and the following N elements

I have an array of sequences of equal length, each sequence contains 300 numbers (M=300). Each element in a sequence is a number from 1 to 9: ...
0
votes
0answers
22 views

Proper way to make Train/test split on Time-Series

I want to create a model with LSTM to predict a user the next purchase value. For this I have used I used a user's purchase history. I have created the model and it works well, but honestly, I don't ...
1
vote
0answers
8 views

LSTM high accuracy but poor generation performance

I'm writing a LSTM model for generating music (in particular drums). My model is based on these 2 models: LSTM text generator LSTM drum generator The model seems to work fine, it trains and I can ...
0
votes
0answers
15 views

LSTM versus Random forest for Time series forecasting

Would you use the same feature vector when forecasting with a LSTMN as a Random Forest? Say features like 'day of week' and 'hour'. Or does the LSTMN learn this by just remembering from previous time ...
1
vote
0answers
22 views

Time series dynamic model: can we also learn time varying models?

I would like to train multivariate time series models with time varying weight information (time-varying relationship between data and labels). My understanding is that for example, autoregressive ...
2
votes
1answer
26 views

Is randomly split dataset to train and test in lstm model reasonable?(human activity recognition)

I have build an lstm model to predict human activity recognition with dataset OPPORTUNITY. I did two experiment with different oder of processing as below, normalized the dataset with minmax scaler, ...
0
votes
0answers
18 views

Test scores are way lower than cross-validation scores

I split my Dataset with 80% of the data for training and 20% for the test in the context of a binary classification task with a very unbalanced dataset. On the training set I do a 3 folds ...
1
vote
1answer
32 views

LSTM performs poorly with monotonically increasing test set values never seen in training. Why?

I have a dataset of approximately monotonically increasing values (in a time-series). I am using keras and LSTM to train the ...
0
votes
0answers
14 views

LSTM Neural Network gets stuck in a specific state when trying to predict new states over many time periods

I have built an LSTM neural network for category, or latent state, prediction. The data is more or less of the form: x1 = continuos number of current record x2 = continuous number of current record x3 ...
0
votes
0answers
15 views

MultiVariate Multi Step forecasting , If I try to model this as regression how can I do it

I am working on project where I want to predict how much customer use my services which translates to Dollar amount. I have data which is having 500 Customers whose monthly usage of services need to ...
0
votes
1answer
22 views

Combining two sequences for text classification

I'm doing text classification on comments posted on articles/stories. The two human-labeled classes are appropriate and not appropriate (not the same as happy/angry or any "sentiment" ...
35
votes
4answers
5k views

Is LSTM (Long Short-Term Memory) dead?

From my own experience, LSTM has a long training time, and does not improve performance significantly in many real world tasks. To make the question more specific, I want to ask when LSTM will work ...
2
votes
1answer
55 views

Weird behaviour in toy RNN (Keras, LSTM)

I'm trying to learn more about RNNs and I'm tackling a toy problem. I'm generating some data that has a pattern, two 1s followed by three 0s which keeps repeating infinitely without any noise. So my ...
1
vote
1answer
20 views

Understanding the output layer formation of an LSTM unit in Keras

I'm struggling to get my head around how the output shape of an LSTM layer formed. How is the output unit value physically implemented in the layer? For example, if I have an input shape of (128 x 6) ...
0
votes
0answers
20 views

Why is my training unstable?

I am training a Variational autoencoder with and without data labels. When I use labels (blue line), validation error decreases with epochs but without labels (orange line) the training is unstable. ...
1
vote
1answer
31 views

I am unable to understand how LSTM is taking my data as input [closed]

X_train.shape, X_test.shape, Y_train.shape, Y_test.shape ((2457, 55, 26), (820, 55, 26), (2457, 3), (820, 3)) X_train is like having 2457 samples of matrix of size ...
1
vote
1answer
47 views

Are RNNs Markovian?

On the one hand, one can argue that they are since "the hidden layer is simply [derived from] the last hidden state and current input". On the other hand, the whole point of RNNs is that &...
0
votes
0answers
8 views

Keras LSTM POS tagger w/ transfer learning (GloVe) — failing to learn?

I've been trying to research how to use Keras to train a POS tagger; specifically I want it to use an LSTM architecture and to use word embeddings, namely, GloVe. I've taken inspiration from two blogs....
0
votes
0answers
17 views

Training an LSTM on multiple distinct batches of time series data

I am running a time series simulation on an electricity power grid simulation package and I want to use this data to train an LSTM to predict the stability of the grid over a given time interval. My ...
0
votes
0answers
13 views

size of LSTM layer and regression on fraction

I need to give prediction on a variable with a large range of values (all positives). I scaled the values between 0 and 1. My first layer is an embedding layer, which it's vocabulary size might change....
0
votes
0answers
16 views

Forecasting sales for thousands of stores individually with multiple features associated

I have data of 2000 stores with associated 145 features(example: ambience, holidays, no. of brands) and their monthly sales for 2 years. It means that for every store I have sales data and other ...
0
votes
1answer
14 views

Why do Dense layers perform better than a mix of Conv Layers, Recurrent Layers on Sentiment Analysis with BERT emebddings?

I have used BERT to make embeddings out of the imdb review dataset and I am trying out some models to check their perfomance on sentiment analysis (0 for the bad reviews and 1 for the good ones). I ...
1
vote
0answers
11 views

Single LSTM, Multiple Output Layers with Different Loss Functions

Would it be possible to have an LSTM that is followed by two output layers, where each output layer computes a different representation and is followed by two different loss functions (i.e. where the ...
2
votes
1answer
23 views

Question about using rolling windows for time series regression

I have say 10 time series which become the 10 features of my model and I train it on these using a rolling window of 6 to predict the following 1 timestep (so t-5 to t to predict t+1). Thus the input ...
0
votes
0answers
17 views

What could explain a loss geting very small quickly in a LSTM network?

I am trying my first LSTM with keras to classify time-dependent data sets. I have created a training and a testing data sets, which I have normalized: ...
2
votes
1answer
26 views

What is the use of the hidden state in an LSTM network

I am training an LSTM network for time series prediction. My understanding so far is that an LSTM network is suitable for time series prediction because it keeps a 'hidden state' which gives the LSTM ...
0
votes
0answers
23 views

Use deep learning to predict the continuation of one time series given another

I have this problem I'm trying to solve.. I have 2 highly correlated time series (lets call them $A$ & $B$), however past a certain date I only have data for A. I would like to use $A$ to predict ...
0
votes
0answers
12 views

Should LSTM data be a sequence?

let me explain what I want to do, I want to predict the trend of the price of something (1 if it increases in the next hour and 0 otherwise). I have gathered tweets about that and grouped them in ...
0
votes
0answers
19 views

Is it possible to use an LSTM for time series classification?

I've read a lot of literature on using LSTM's for time-series prediction in the regression sense; using past values to predict the next value in a time series.[1][2] However, I have not come across ...
0
votes
0answers
26 views

How exactly does conv1d filter work when operating on a sequence of characters?

I understand convolution filters when applied to an image (e.g. an 224x224 image with 3 in-channels transformed by 56 total filters of 5x5 conv to a 224x224 image with 56 out-channels). The key is ...
1
vote
0answers
16 views

Training and Testing LSTM [closed]

I've been reading this article about multivariate LSTM. I'm still a beginner so my questions might be silly. As far as I understand, we need to divide the data set on training and testing/validation. ...
0
votes
1answer
67 views

When computing parameters, why is dimensions of hidden-output state of an LSTM-cell assumed same as the number of LSTM-cell?

I was trying to figure out how to estimate the number of parameters in an LSTM layer. What is the relationship of number of parameters with the num lstm-cells, input-dimension, and hidden output-state ...
1
vote
0answers
22 views

Multivariate time series classification/event detection

I have a 1.5 million row mutivariate time series dataset that looks like this: ...
1
vote
0answers
10 views

Scaling test set based on training will cause test set to have values greater than the scale

I have a time series data that does not have an upper limit (data is somewhat monotonically increasing). Making the Test set values larger than the training set. (I am not shuffling because time ...
0
votes
0answers
7 views

Large amount of missing values in as input features for LSTM time series

I am using an LSTM to predict a time series chart from multiple other time series charts as input features. The problem is that some of these input charts have much ...

1
2 3 4 5
13