Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
0
votes
0answers
4 views
Is negative Viterbi Loss possible?
So I'm training a sequence-labeling model with a BiLSTM-CRF architecture, and I am getting negative values on Viterbi Loss. Is this possible?
I'm using the following formula in my code, as specified ...
0
votes
0answers
7 views
Resetting states in tf.kers.layers.RNN to different batchsize [on hold]
I am trying to use a stateful RNN in Tensorflow Eager (link). For my validation set, in the end I have leftover sequences which arent a full batch. To reset the state for new sequences, I normally use ...
1
vote
1answer
15 views
How to infer state space parameters from an LSTM model?
I'm attempting to create a state-space model by training my time series data with an LSTM. I'm hoping the LSTM will capture non-linear phenomenon as opposed to a linear state-space model. The only ...
0
votes
0answers
65 views
Help understanding if suffering from Validation Bias
The goal is to forecast the volume a product will sell in future months.
There are about 107 products that are being bought by different customers for different uses.
It is univariate problem since ...
0
votes
0answers
4 views
Which feature representation is better? MFCC or Mel-filter bank?
Sometimes neural network based technique posses good accuracy with Mel-filter bank compare to MFCC. But why? I was studying about it and found those points.
The Mel-filter bank parameters are highly ...
0
votes
0answers
7 views
Would it make sense to use an LSTM to predict a list of events that occur at a specific time
I have a dataset that for every hour of an entire month, contains a set of events that occur during the hour. These events are not predefined and can be any string, and are definitely not categorical ...
0
votes
0answers
43 views
How exactly keras LSTM layer works?
I try to create a sentiment analysis that have 7 classification. Let's say, I have 100.000 unique word (already converted into 100.000 integer) which have the longest input is 41. I created 3 layer ...
0
votes
1answer
14 views
Transformer based decoding
Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder ...
0
votes
0answers
28 views
LSTM Autoencoders - architecture
I am a bit confused about the structure of LSTM autoencoders, as far as I know, common way to construct vanilla autoencoders is bottleneck structure, for instance, start with 40 nodes, encode it to 30 ...
0
votes
0answers
12 views
What to use as a “stop” signal in a hierarchical LSTM model with continuous variable-length outputs
I am implementing a hierarchical sequence-to-sequence deep neural network model using long short-term memory (LSTM), where the bottom level of the hierarchy generates discrete outputs (characters from ...
0
votes
0answers
19 views
Recurrent Neural Network - Time Series prediction: How to decide on the sequence length and number of output neurons
Context
I am currently working with Gated Recurrent Units for time series prediction. Anyhow, my question should apply to all kinds of such recurrent networks.
In my data, I have timestamps of 15 ...
1
vote
1answer
32 views
Predict song genre using LSTM
I have a dataset of songs based on genres. For example, a song may hold {5, 2, 3} as scores set for Sentimental, Rock and Jazz. In total there are 800 songs sequentially arranged. I want to predict ...
0
votes
0answers
13 views
How do I implement masking in TensorFlow eager?
I am training a stateful RNN on variable length sequences (optional: see my previous question for more details).
I padded the sequences to a fixed length with the value -1.
The when batches are ...
0
votes
1answer
27 views
How to do location forecasting on Chicago Crime Dataset?
I am using the dataset https://www.kaggle.com/currie32/crimes-in-chicago and given primary type of the crime I want to forecast the next location of crime. What approach should I follow ?
0
votes
1answer
32 views
Given the universal approximation theorem, why are LSTM better than feed forward neural networks at certain tasks?
Per the universal approximation theorem, feed forward neural networks can approximate any function up to an arbitrary level of precision on the domain that they are trained on, given a sufficient ...
0
votes
0answers
29 views
Sequence Classification Machine Learning Methods
I know that this question might have been asked before but I did not find any response so far. My problem is that I am doing Multi-class sequence classification with Stacked LSTM (3 layers) using ...
1
vote
0answers
13 views
LSTM repeatedly giving the same words
I implemented a 2-layer LSTM based on the architecture described in https://panderson.me/up-down-attention/
I trained it with Cross Entropy, and for every step I fed in the correct previous word (...
0
votes
0answers
7 views
Training and validation loss: consistency and interpretation
I have following training & validation loss for my LSTM network.
I was wondering what I could deduce from this data. The validation loss seems to start where the training loss ends, is this a ...
0
votes
0answers
19 views
Is it okay to calculate all the gradients for an LSTM at once?
I'm trying to use the AC method reported in "Asynchronous Methods for Deep Reinforcement Learning" for a project. The relevant algorithm is shown in pseudocode at the bottom, Algorithm S3.
I'm using ...
0
votes
1answer
20 views
What is the output of an LSTM
I have two questions regarding LSTMs:
1) Are LSTMs outputs' have shape/size exactly similar to the input?
2) Can we use LSTMs' intermediate outputs to deduce some sort of predictions?
Context:
...
0
votes
0answers
10 views
Using RNNs to segment long streams
I'm using an LSTM to segment a long streams of linear data into contiguous parts. My specific application is extracting key signature changes from a stream of notes. Specifically, I want to (for each ...
1
vote
1answer
45 views
Examples of “one to many” for RNN/LSTM
Are there any examples dealing with "one to many" kind of LSTM?
Basically I am trying to build a model which takes an input vector $a$ and gives an output of $[b_1; b_2 ;b_3; b_4, \ldots; b_n]$ where ...
0
votes
0answers
8 views
Simple Keras LSTM model does not converge [duplicate]
I try to predict time-series with simple Keras LSTM model:
...
4
votes
0answers
30 views
Feature engineering for sheet music
I have a large dataset of digitized music scores that I'd like to use as input to a network. Initially, I'm looking to train networks to identify key signatures, tempo, dynamics, etc. from the raw ...
0
votes
0answers
23 views
Understanding epoch, batch size, accuracy ,performance gain in lstm forecasting model
I am new to machine learning and lstm. I am referring this link LSTM for multistep forecasting for Encoder-Decoder LSTM Model With Multivariate Input
section.
Here ...
0
votes
0answers
17 views
RNN to predict completion of fixed-length time series
I need advice to model a certain kind of time series prediction for which I didn't find any existing solution. I have a large set of independent time series of fixed length (let's say 100 steps). All ...
0
votes
0answers
25 views
Tree-Paths as sequence input into a neural network
I'm currently trying to understand this paper but I struggle with the input into the NN. What I don't understand is what the input vectors should look like for the network described in b) in the image ...
0
votes
0answers
47 views
LSTM - When to use sliding window in time series classification?
Say I have a tensor of data with shape (30, 16000, 38) - where each tuple element corresponds to ...
0
votes
0answers
8 views
when not using teacher forcing does backpropagation flow through input layers to output layers?
When training a Recurrent Neural Network that feeds outputs in as the next input(no teacher forcing) do gradients flow through input layers to output layers and also through the hidden states?
0
votes
0answers
33 views
Attention mechanism in LSTM model to predict next action
I have a dataset. Each data point of this set contains a variable length of sequences with 7 letters. For example, Data point 1 has a sequence (A, A, B, E, B, C, D, E, D.....). I used LSTM to predict ...
0
votes
1answer
65 views
Binary Classification of Numeric Sequences with Keras and LSTMs [duplicate]
I'm attempting to use a sequence of numbers (of fixed length) in order to predict a binary output (either 1 or 0) using Keras and a recurrent neural network.
Each training example/sequence has 10 ...
2
votes
0answers
18 views
LSTM Generates Duplicates
I'm using an LSTM to generate molecules using the Simplified Molecular Input Line Entry System (SMILES) to represent molecules. As an example, Aspirin is represented as O=C(C)Oc1ccccc1C(=O)O
I have ...
0
votes
0answers
20 views
Representative Pattern Extraction from Time Series using LSTM
I am interested in extracting a representative pattern from time series having variable time periods. I have attached an image for reference. I would like to know if a LSTM would be the appropriate ...
0
votes
0answers
18 views
Why my LSTM model cannot predict waveform
Basically, I'm trying to use one waveform (above) to predict another waveform ().
And Here are my model structure:
...
2
votes
2answers
100 views
How does an LSTM process sequences longer than its memory?
* Note: The premise of my question was incorrect in the first place. My question assumes that an LSTM maintains a separate set of weights for each time step in the memory it is given as a design ...
0
votes
0answers
12 views
Adding context in a seq2seq RNN model
The encoder of a seq2seq model is meant to generate a conditioning context for the decoder, as mentioned here
A RNN layer (or stack thereof) acts as "encoder": it processes the input sequence and ...
1
vote
0answers
58 views
Why is accuracy very low and losses high and fluctuating for cnn-lstm
Below given is my cnn-lstm architecture.
...
1
vote
1answer
157 views
LSTM - Multiple Time Series, degrading accuracy
I'm trying to make a LSTM model for detecting failures on a physical system, by supplying 27 features of sensor data. I've inputted three disjunct timeseries, each beginning with "normal" operational ...
0
votes
1answer
34 views
How is Bidirectional-RNN different from vanilla RNN trained with both original & reverse copies of data?
I have several questions regarding Bi-RNN. The RNN here can be LSTM or GRU as well.
(1) What is the input of Bi-RNN when making inference? For RNN, if I want to predict a $\hat{y}(t)$ for the target $...
0
votes
1answer
14 views
Product Price Prediction - using online scrapped data [closed]
AIM: To Predict Price of products based on data that I have taken from other online stores. e.g Predict price of Samsung Galaxy S10, data will be from multiple online stores.
Problem: Which Machine ...
4
votes
3answers
102 views
Binary target prediction using LSTM with sparse events in time
I have a data of patients that have multiple events happening in there medical history, I'd like to predict a target of having a specific targeted-event in the next 30 days.
The data is timestamped ...
0
votes
0answers
16 views
What might explain inferior performance in a LSTM featuring a convolution layer?
This is not a problem:solution scenario insofar that I am not attempting to find a way to improve the model, merely find a reason for its behaviour.
Model using LSTM has accuracy of about 84-86% ...
0
votes
0answers
45 views
how to prepare text data for LSTM autoencoder
My main goal is to come up with some topics using LSTM autoencoder.
I want to use 20 news_group data set.
after reading lots of material and looking at some GitHub project, I am still not clear how ...
0
votes
0answers
20 views
Shapes of input and outputs for LSTM architecture?
I have a sequence data like
X1, X2, X3, X4, X5 -> y1,y2,y3,y4,y5
X6,X7,X8 -> y6, y7, y8
Where Xi is m x n dimension matrix, n is the number of columns (...
1
vote
1answer
74 views
LSTM time series forecasting on sparse dataset
I am working on the LSTM time series forecasting of solar energy production. The available data is one year on a half hourly basis. More than 60% of the data values are zero as the PV stations cannot ...
0
votes
0answers
18 views
How to model a time series problem for RNNs?
I have projects as input data, each project has a weekly progress report (hours of work completed in this week). A project can have an arbitrary duration, but let's say it's usually around 100 weeks, ...
3
votes
1answer
188 views
Cannot understand LSTM inference
I seem to have stumbled on a hole in my understanding around LSTMs. In short, I cannot understand how even a simple one is actually fed samples, upon inference time/training time. Here are the details:...
0
votes
0answers
34 views
Small output range and delayed output? Predicting sine using LSTM
I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict 0.5*sin(t + N) given 0.5*sin(t) as an exercise.
I have tweaked the model, changing the output layer ...
0
votes
1answer
33 views
Attention for short sequence length. Is it reasonable?
Will the attention mechanism be useful for the short sequence length?
Let's say your training corpus has each query of MAX length 10. and most queries are of word length 3-4 words. How reasonable is ...
0
votes
0answers
39 views
Do we get the best performance with “batch_size = 1”(especially for LSTM)?
In my experience, choosing batch_size = 1 gives the best result and choosing the batch_size = whole data number gives the worst. ...
