Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
0
votes
0answers
8 views
How to get consistency in neural network and eliminate possibility of NaN values?
I'm using a neural network(Keras,LSTM) for time series regression. Whenever I run the network, I get different outputs for the prediction. This is presumably due to the randomised weight ...
2
votes
0answers
16 views
LSTM : multi-step multidimensional multivariate multi-site timeseries forecasting [closed]
I'm working on a project in which i'm trying to do a pollution forecasting.
I googled around and found that LSTM is a good candidate for this task, however, I'm still struggling at how to adapt it to ...
2
votes
1answer
36 views
Forecasting algorithms for incomplete time series data [duplicate]
I want to forecast the demand of each SKU in my warehouse every week from the history transaction that I have collected. The data contains brand, product type, SKU, quantity, date(per day), price. But ...
0
votes
0answers
7 views
How to paraphrase and augment training data for a question answering ML model?
I have only 50 question, answer pairs in my training data, where each question represent a unique intent. However, the training data is too small to build any meaningful ML model.
What are the ...
0
votes
0answers
22 views
Using LSTMs for Time series prediction for quantities varying with different time units
I am trying to implement neural networks - LSTMs - to implement a Time Series concept. I have 3 years of hourly data of production and I plan to use 2 years of data to train the model and predict for ...
0
votes
0answers
12 views
Input shape for LSTM to predict customer usage quantity for a given timestep
As per my understanding the Input shape for RNN must be (Number of samples, number of timesteps, number of features). My data has 12 timesteps for each customer and number of features is 15, my ...
0
votes
1answer
42 views
LSTM output dimensionality
I am new to LSTMs. When reading the papers and websites about LSTM architecture, there is something I do not get.
As I understand it, a single LSTM layer can have multiple LSTM cells (just like a ...
1
vote
0answers
26 views
How to apply dropout in LSTMs?
Dropout in fully connected neural networks is simpl to visualize, by just 'dropping' connections between units with some probability set by hyperparamter p.
However, how dropout works in recurrent ...
0
votes
0answers
30 views
Reduce the effect of excessive zeros
I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I ...
0
votes
0answers
5 views
Are weights shared in different unfolded block of LSTM? [duplicate]
I am new to LSTM. For the unfolded LSTM block shown above, weights in different block($h_{t-1}$, $h_t$, $h_{t+1}$) are the same, is that right?
1
vote
0answers
27 views
Getting over bid-ask bounce
High-frequency financial data is subject to bid-ask bounce.
Description : Unlike traditional data based on just closing prices, tick data carry additional supply-and-demand information in the form of ...
1
vote
0answers
18 views
Limiting regression accuracy
I am currently using a LSTM Network to solve a regressional problem. The goal is to predict the payload mass using various time series as input data. For our intents and purposes estimating the mass ...
1
vote
0answers
30 views
Zero inflated binomial for excessive zeros
"What are some tricks for dealing with a zero inflated response variable when tackling a machine learning regression problem?"
Answer :
"One of the easiest and most intuitive methods is to run a ...
1
vote
2answers
41 views
Deep learning methods with seasonal data
I have been building an LSTM model in Python which will predict the number of passengers arriving at a station in the next 15 mins.
My dataset is arrivals at the station every 15 mins across a 50 day ...
0
votes
1answer
75 views
Time steps in Keras LSTM
My understanding of time-series LSTM training is that the recurrent cell gets unrolled to a specified length (num_steps), and parameter updates are back-propagated ...
0
votes
3answers
110 views
Dealing with excessive number of zeros
ipdb> np.count_nonzero(test==0) / len(ytrue) * 100
76.44815766923736
...
0
votes
2answers
47 views
Why is LSTM so ineffective at this ridiculously simple sequence?
I am trying to predict the the sequence $(0.1, 0.05, …, 1.1)$ from the sequence $(0, 0.05, 0.1, …, 1)$. I thought this would be the only item in a toy dataset.
Now I implemented a long-short term ...
1
vote
0answers
39 views
Adding noise to time series data to increase training data
I am dealing with a weekly time series forecasting problem and I am currently investigating the use of an LSTM to make a multi-step forecast for a univariate time series. I actually have a ...
0
votes
0answers
40 views
What are paralell training and attention mechanism?
I read a quite interesting paper here:
http://hanj.cs.illinois.edu/pdf/kdd18_cyang.pdf
Accordingly, the basic idea is to combine clustering and churn prediction so that it can imply some insight from ...
0
votes
0answers
22 views
Getting the last hidden states of bi-LSTM in variable-length batch in PyTorch
According to the PyTorch manual, if you want to feed a batch of variable-length sequences to a sequence model, you pad and pack the batch before feeding it to the sequence model. That is,
...
0
votes
1answer
90 views
Keras TimeSeries - Regression with negative values
I am trying to make regression tasks for time series, my data is like the below, i make window size of 10, and input feature as below, and target is the 5th column. as you see it has data of {70, 110, ...
0
votes
0answers
20 views
LSTM Tuning, Predictions, Multiple feature input
I have a few questions wrapped up into one post
I am using this simple example found online to work with LSTM models but I would like to utilize all of the features instead of just one.
What is the ...
0
votes
0answers
16 views
Query on LSTM (Request for guidance)
I recently started working on a project at the University.
The main task of the project is to apply Deep Learning for forecasting. I have the dataset from a company that basically contains various ...
0
votes
0answers
88 views
Price difference predictions curve almost vanished [duplicate]
With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well ...
0
votes
0answers
17 views
LSTM & GRU layer for sound classification
I have a problem at hand that involves classifying sound samples as belonging to 5 different classes.
The input shape is (samples, features) = (260000, 108), in this case time steps and 108 frequency ...
1
vote
1answer
41 views
Regression on price or price difference
I am trying to predict the price movement for high-frequency trading purpose, but so far I did not get a lot of success with just predicting the price directly.
We can see that there's a little ...
1
vote
1answer
64 views
Price Forecasting Problem
I am working on a project for price movement forecasting and I am stuck with poor quality predictions.
At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the ...
0
votes
1answer
54 views
How to deal with really sparse time series data for a binary classification task using RNN or LSTM?
I have a binary classification prediction task and more often than not, the time series data is like really sparse. The number of zeroes in the time series data is almost always more than 99%. I ...
1
vote
0answers
17 views
How does the internal state of Tensorflow's “dtf.nn.dynamic_rnn()” change?
I am trying to do a uni-variate forecasting and I have come across this question quite a lot. This question is regarding TensorFlow's "dtf.nn.dynamic_rnn()".
NOTE: When I say internal state I am ...
0
votes
0answers
32 views
Autoencoder with small dataset - simple images
I wanted to ask a simple question regarding autoencoders to parse for tips and possible advice before diving into this path.
I have a small dataset ~50-100 heatmap images that are relatively simple.
...
1
vote
1answer
34 views
How to minimize sharpe ratio with LSTM recurrent neural network?
I've read some articles about trading using recurrent reinforcement learning such as this one. The point where I do not fully understand is how to construct the cost/loss function.
In the article, ...
0
votes
0answers
12 views
Keras gradient flow to compare two models
I am trying to compare two models for the imbalanced dataset. I am having simple LSTM F score of 0.51 vs other LSTM with attention mechanism F score of 0.49. In my opinion the F score should have ...
1
vote
1answer
21 views
Which activation function is better to a 1-dimensional time series in a LSTM model?
I am experimenting with a LSTM model (I have normalized the data) util now the 'sigmoid' performs better than others. How can I justify/interpret it?
0
votes
0answers
13 views
Mean Test Set Performance of LSTM & Evaluation
In the paper "Greff et al, 2017 - LSTM A Search Space Odyssey" they evaluate different variants of LSTM architectures against different tasks/datasets.
Could you help me to understand the evaluation?
...
0
votes
1answer
23 views
Testing an LSTM making predictions 1 timestep into the future
Say I have a time series data set of 100 sequential timesteps, and I want to train and test an LSTM on the data set, but only on forecasting a single timestep into the future.
I want more than one ...
0
votes
0answers
10 views
Support Vector Machines VS LSTMs: How well it is justifiable to use LSTM for its Generalization properties?
The question is pretty straightforward, How well one can justify using LSTMs(Neural Networks) for text classification task in terms of "Generalization" compared to classic support vector machines(SVM) ...
1
vote
2answers
51 views
How to define a time series classification problem?
I have 3 sets of time series data generated from sensors, I believe they have some correlation themselves. Certain "modes" of the system can be defined from the patterns from these signals. The signal ...
0
votes
1answer
26 views
Deep learning with a lot of training data
I am building a bidirectional LSTM to do a sequential text-tagging task (particularly, automatic punctuation). Usually, the training is done in iterations, where in each iteration, the entire training ...
1
vote
0answers
18 views
Interpreting accuracy graph for a LSTM model | Keyword Prediction
I have created a LSTM model for keyword prediction. I am using RMSE optimizer for training. I observe that the train and test accuracy decreases at first and then fluctuates without much difference in ...
0
votes
0answers
31 views
Best practices to apply Layer normalization in recurrent networks
I'm trying to add layer normalization (in the encoder-level) to the Listen-attend-and-spell model for speech recognition tasks.
To do so, I have done many experiments (all of them failed) to make my ...
0
votes
0answers
43 views
NaN output from Bidirectional LSTM in Keras
I am creating a bi-directional LSTM using tf.keras APIs:
input_layer = tf.reshape(emb_seqs, [const.TRN_BATCH_SIZE, -1, const.VECTOR_SIZE])
lstm_layer = tf.keras.layers.LSTM(units=LSTM_UNITS, ...
1
vote
0answers
43 views
How does LSTM learn [duplicate]
I am confused on how LSTM learn from word embedding. I know that LSTM accepts 3D input (sample, timesteps, features). So, when we use embedding layer (word2vec) and we have 300-d vector representation,...
0
votes
0answers
26 views
How to understand what is going on based on the loss curves?
I have been implementing a many-to-one LSTM and have been searching for the best hyperparameters. However, sometimes I am a little confused on the reason the loss acts as it does. Here is a screenshot ...
1
vote
1answer
17 views
Getting ValueError while implementing LSTM in keras
I am getting this error while implementing LSTM in Keras:
""""Error when checking input: expected lstm_16_input to have 3 dimensions, but got array with shape (156060, 1)""""
I have 156060 text ...
0
votes
1answer
9 views
Clarification about RNN/LSTM Sequence Models with Word Vector inputs
Say that we are trying to train a language model with an RNN/LSTM i.e. the inputs are words in a sentence and the outputs are the same words shifted by one such that for each input word the output is ...
0
votes
0answers
27 views
How to overfit an LSTM for acoustic features?
Aim
I would like to train an LSTM to learn the mapping between the EMA signals and audio signals in the MOCHA-TIMIT dataset. I've looked at publications using similar approaches to see if I can ...
0
votes
0answers
15 views
Different ways to deepen LSTMs
Quick introduction of the notation I'm working with:
With regular RNNs we have
$$h^{\left\langle t\right\rangle }=g_{h}\left(W_{h}\left[h^{\left\langle t-1\right\rangle },x^{\left\langle t\right\...
0
votes
0answers
102 views
Loss decreasing but highly oscillating over all training examples
I'm training a sequence model. I have 40 sequences of length 70,000. I have divided these into minibatches of size $[40, 200]$. Since I want to utilise the stateful nature of the GRU I'm training, I ...
4
votes
1answer
123 views
State-of-the-art algorithms for the training of neural networks with GRU or LSTM units
I recently read a lot about neural networks using GRU or LSTM units. There are many easy to use frameworks like tensorflow that do not even require high knowledge about programming. Unfortunately, I ...
1
vote
0answers
60 views
Getting OSerror while loading my model in keras [closed]
I created a model in keras and then save it to h5 file then I created a package for that model and used load_model function of keras to load that saved model of mine but I am getting this error:
""...
