Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
0
votes
0answers
7 views
Transformer based decoding
Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder ...
0
votes
0answers
24 views
LSTM Autoencoders - architecture
I am a bit confused about the structure of LSTM autoencoders, as far as I know, common way to construct vanilla autoencoders is bottleneck structure, for instance, start with 40 nodes, encode it to 30 ...
0
votes
0answers
10 views
What to use as a “stop” signal in a hierarchical LSTM model with continuous variable-length outputs
I am implementing a hierarchical sequence-to-sequence deep neural network model using long short-term memory (LSTM), where the bottom level of the hierarchy generates discrete outputs (characters from ...
0
votes
0answers
17 views
Recurrent Neural Network - Time Series prediction: How to decide on the sequence length and number of output neurons
Context
I am currently working with Gated Recurrent Units for time series prediction. Anyhow, my question should apply to all kinds of such recurrent networks.
In my data, I have timestamps of 15 ...
0
votes
0answers
6 views
LSTM/RNN history-based prediction by using Keras backend?
For my experiment, I have a formatted csv file with 1440 columns like following:
...
1
vote
1answer
31 views
Predict song genre using LSTM
I have a dataset of songs based on genres. For example, a song may hold {5, 2, 3} as scores set for Sentimental, Rock and Jazz. In total there are 800 songs sequentially arranged. I want to predict ...
0
votes
0answers
13 views
How do I implement masking in TensorFlow eager?
I am training a stateful RNN on variable length sequences (optional: see my previous question for more details).
I padded the sequences to a fixed length with the value -1.
The when batches are ...
0
votes
1answer
25 views
How to do location forecasting on Chicago Crime Dataset?
I am using the dataset https://www.kaggle.com/currie32/crimes-in-chicago and given primary type of the crime I want to forecast the next location of crime. What approach should I follow ?
0
votes
1answer
28 views
Given the universal approximation theorem, why are LSTM better than feed forward neural networks at certain tasks?
Per the universal approximation theorem, feed forward neural networks can approximate any function up to an arbitrary level of precision on the domain that they are trained on, given a sufficient ...
0
votes
0answers
26 views
Sequence Classification Machine Learning Methods
I know that this question might have been asked before but I did not find any response so far. My problem is that I am doing Multi-class sequence classification with Stacked LSTM (3 layers) using ...
1
vote
0answers
13 views
LSTM repeatedly giving the same words
I implemented a 2-layer LSTM based on the architecture described in https://panderson.me/up-down-attention/
I trained it with Cross Entropy, and for every step I fed in the correct previous word (...
0
votes
0answers
7 views
Training and validation loss: consistency and interpretation
I have following training & validation loss for my LSTM network.
I was wondering what I could deduce from this data. The validation loss seems to start where the training loss ends, is this a ...
0
votes
0answers
19 views
Is it okay to calculate all the gradients for an LSTM at once?
I'm trying to use the AC method reported in "Asynchronous Methods for Deep Reinforcement Learning" for a project. The relevant algorithm is shown in pseudocode at the bottom, Algorithm S3.
I'm using ...
0
votes
1answer
17 views
What is the output of an LSTM
I have two questions regarding LSTMs:
1) Are LSTMs outputs' have shape/size exactly similar to the input?
2) Can we use LSTMs' intermediate outputs to deduce some sort of predictions?
Context:
...
0
votes
0answers
7 views
Using RNNs to segment long streams
I'm using an LSTM to segment a long streams of linear data into contiguous parts. My specific application is extracting key signature changes from a stream of notes. Specifically, I want to (for each ...
1
vote
1answer
28 views
Examples of “one to many” for RNN/LSTM
Are there any examples dealing with "one to many" kind of LSTM?
Basically I am trying to build a model which takes an input vector $a$ and gives an output of $[b_1; b_2 ;b_3; b_4, \ldots; b_n]$ where ...
0
votes
0answers
8 views
Simple Keras LSTM model does not converge [duplicate]
I try to predict time-series with simple Keras LSTM model:
...
4
votes
0answers
29 views
Feature engineering for sheet music
I have a large dataset of digitized music scores that I'd like to use as input to a network. Initially, I'm looking to train networks to identify key signatures, tempo, dynamics, etc. from the raw ...
0
votes
0answers
21 views
Understanding epoch, batch size, accuracy ,performance gain in lstm forecasting model
I am new to machine learning and lstm. I am referring this link LSTM for multistep forecasting for Encoder-Decoder LSTM Model With Multivariate Input
section.
Here ...
0
votes
0answers
16 views
RNN to predict completion of fixed-length time series
I need advice to model a certain kind of time series prediction for which I didn't find any existing solution. I have a large set of independent time series of fixed length (let's say 100 steps). All ...
0
votes
0answers
22 views
Tree-Paths as sequence input into a neural network
I'm currently trying to understand this paper but I struggle with the input into the NN. What I don't understand is what the input vectors should look like for the network described in b) in the image ...
0
votes
0answers
30 views
LSTM - When to use sliding window in time series classification?
Say I have a tensor of data with shape (30, 16000, 38) - where each tuple element corresponds to ...
0
votes
0answers
7 views
when not using teacher forcing does backpropagation flow through input layers to output layers?
When training a Recurrent Neural Network that feeds outputs in as the next input(no teacher forcing) do gradients flow through input layers to output layers and also through the hidden states?
0
votes
0answers
29 views
Attention mechanism in LSTM model to predict next action
I have a dataset. Each data point of this set contains a variable length of sequences with 7 letters. For example, Data point 1 has a sequence (A, A, B, E, B, C, D, E, D.....). I used LSTM to predict ...
0
votes
1answer
45 views
Binary Classification of Numeric Sequences with Keras and LSTMs [duplicate]
I'm attempting to use a sequence of numbers (of fixed length) in order to predict a binary output (either 1 or 0) using Keras and a recurrent neural network.
Each training example/sequence has 10 ...
2
votes
0answers
18 views
LSTM Generates Duplicates
I'm using an LSTM to generate molecules using the Simplified Molecular Input Line Entry System (SMILES) to represent molecules. As an example, Aspirin is represented as O=C(C)Oc1ccccc1C(=O)O
I have ...
0
votes
0answers
17 views
Representative Pattern Extraction from Time Series using LSTM
I am interested in extracting a representative pattern from time series having variable time periods. I have attached an image for reference. I would like to know if a LSTM would be the appropriate ...
0
votes
0answers
16 views
Why my LSTM model cannot predict waveform
Basically, I'm trying to use one waveform (above) to predict another waveform ().
And Here are my model structure:
...
2
votes
2answers
93 views
How does an LSTM process sequences longer than its memory?
* Note: The premise of my question was incorrect in the first place. My question assumes that an LSTM maintains a separate set of weights for each time step in the memory it is given as a design ...
0
votes
0answers
11 views
Adding context in a seq2seq RNN model
The encoder of a seq2seq model is meant to generate a conditioning context for the decoder, as mentioned here
A RNN layer (or stack thereof) acts as "encoder": it processes the input sequence and ...
1
vote
0answers
48 views
Why is accuracy very low and losses high and fluctuating for cnn-lstm
Below given is my cnn-lstm architecture.
...
1
vote
1answer
124 views
LSTM - Multiple Time Series, degrading accuracy
I'm trying to make a LSTM model for detecting failures on a physical system, by supplying 27 features of sensor data. I've inputted three disjunct timeseries, each beginning with "normal" operational ...
0
votes
1answer
31 views
How is Bidirectional-RNN different from vanilla RNN trained with both original & reverse copies of data?
I have several questions regarding Bi-RNN. The RNN here can be LSTM or GRU as well.
(1) What is the input of Bi-RNN when making inference? For RNN, if I want to predict a $\hat{y}(t)$ for the target $...
0
votes
1answer
14 views
Product Price Prediction - using online scrapped data [closed]
AIM: To Predict Price of products based on data that I have taken from other online stores. e.g Predict price of Samsung Galaxy S10, data will be from multiple online stores.
Problem: Which Machine ...
4
votes
3answers
92 views
Binary target prediction using LSTM with sparse events in time
I have a data of patients that have multiple events happening in there medical history, I'd like to predict a target of having a specific targeted-event in the next 30 days.
The data is timestamped ...
0
votes
0answers
15 views
What might explain inferior performance in a LSTM featuring a convolution layer?
This is not a problem:solution scenario insofar that I am not attempting to find a way to improve the model, merely find a reason for its behaviour.
Model using LSTM has accuracy of about 84-86% ...
0
votes
0answers
35 views
how to prepare text data for LSTM autoencoder
My main goal is to come up with some topics using LSTM autoencoder.
I want to use 20 news_group data set.
after reading lots of material and looking at some GitHub project, I am still not clear how ...
0
votes
0answers
19 views
Shapes of input and outputs for LSTM architecture?
I have a sequence data like
X1, X2, X3, X4, X5 -> y1,y2,y3,y4,y5
X6,X7,X8 -> y6, y7, y8
Where Xi is m x n dimension matrix, n is the number of columns (...
0
votes
1answer
56 views
LSTM time series forecasting on sparse dataset
I am working on the LSTM time series forecasting of solar energy production. The available data is one year on a half hourly basis. More than 60% of the data values are zero as the PV stations cannot ...
0
votes
0answers
18 views
How to model a time series problem for RNNs?
I have projects as input data, each project has a weekly progress report (hours of work completed in this week). A project can have an arbitrary duration, but let's say it's usually around 100 weeks, ...
3
votes
1answer
159 views
Cannot understand LSTM inference
I seem to have stumbled on a hole in my understanding around LSTMs. In short, I cannot understand how even a simple one is actually fed samples, upon inference time/training time. Here are the details:...
0
votes
0answers
23 views
Small output range and delayed output? Predicting sine using LSTM
I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict 0.5*sin(t + N) given 0.5*sin(t) as an exercise.
I have tweaked the model, changing the output layer ...
0
votes
1answer
32 views
Attention for short sequence length. Is it reasonable?
Will the attention mechanism be useful for the short sequence length?
Let's say your training corpus has each query of MAX length 10. and most queries are of word length 3-4 words. How reasonable is ...
0
votes
0answers
28 views
Do we get the best performance with “batch_size = 1”(especially for LSTM)?
In my experience, choosing batch_size = 1 gives the best result and choosing the batch_size = whole data number gives the worst. ...
0
votes
1answer
138 views
Parameters Grid Search for Keras LSTM on Time Series
How do you do grid search for Keras LSTM on time series? I have seen various possible solutions, some recommend to do it manually with for loops, some say to use scikit-learn GridSearchCV. Feedback ...
1
vote
1answer
117 views
LSTM state-of-art [closed]
Can you advice me a good paper which talks about the LSTM's state-of-art?
I have already searched on google but I have not found anything interesting
Thank you very much
0
votes
0answers
13 views
Is it possible to make LSTM model with 4dimension shape?
Hellow, wizards. I have time series data including sevaral days.
I try to predict a grade of tomorrow, which is range from 0 to 100. And I assume that this grade depends on 3 time-series independent ...
0
votes
0answers
335 views
Number of Hidden Layer Nodes in Recurrent Neural Networks
There's already a decent discussion on how to select the right number of hidden layers and hidden nodes in a feed-forward neural network: How to choose the number of hidden layers and nodes in a ...
0
votes
0answers
113 views
Time-series classification of Kinect data using Keras
For my PhD project I recorded using Kinect and Myo 11 people performing Cardiopulmonary Resuscitation (CPR), repeatedly doing chest compressions to a manikin (one person per time). I collected in ...
2
votes
0answers
55 views
Time Series Forecasting RNN: Difference Between Masking & Excluding Rows
Suppose you have missing values in a time series E.g. :
t1 x1 y1
t2 ? ?
t3 x3 y3
t4 ? ?
t5 x5 y5
You are trying to forecast this time series using a recurrent ...
