All Questions
Tagged with neural-networks time-series
419 questions
92
votes
1
answer
88k
views
How to apply Neural Network to time series forecasting?
I'm new to machine learning, and I have been trying to figure out how to apply neural network to time series forecasting. I have found resource related to my query, but I seem to still be a bit lost. ...
- 1,055
70
votes
3
answers
48k
views
Proper way of using recurrent neural network for time series analysis
Recurrent neural networks differ from "regular" ones by the fact that they have a "memory" layer. Due to this layer, recurrent NN's are supposed to be useful in time series modelling. However, I'm not ...
- 2,767
55
votes
5
answers
41k
views
Using deep learning for time series prediction
I'm new in area of deep learning and for me first step was to read interesting articles from deeplearning.net site. In papers about deep learning, Hinton and others mostly talk about applying it to ...
- 651
34
votes
1
answer
6k
views
What are attention mechanisms exactly?
Attention mechanisms have been used in various Deep Learning papers in the last few years. Ilya Sutskever, head of research at Open AI, has enthusiastically praised them:
https://towardsdatascience....
- 18.4k
30
votes
3
answers
30k
views
Hidden Markov Model vs Recurrent Neural Network
Which sequential input problems are best suited for each? Does input dimensionality determine which is a better match? Are problems which require "longer memory" better suited for an LSTM RNN, while ...
- 868
22
votes
2
answers
24k
views
Convolutional neural network for time series? [closed]
I would like to know if there exists a code to train a convolutional neural net to do time-series classification.
I have seen some recent papers (http://www.fer.unizg.hr/_download/repository/KDI-...
- 391
21
votes
3
answers
5k
views
Why is this prediction of time series "pretty poor"?
I am trying to learn how to use Neural Networks. I was reading this tutorial.
After fitting a Neural Network on a Time Series using the value at $t$ to predict the value at $t+1$ the author obtains ...
- 2,286
19
votes
2
answers
3k
views
Showing machine learning results are statistically irrelevant
This is a question as part of a paper review which was already published. The authors of the paper publish $R^2$ and RMSE in training but only RMSE in validation. Utilizing the published code, $R^2$ ...
- 315
17
votes
3
answers
4k
views
Why back propagate through time in a RNN?
In a recurrent neural network, you would usually forward propagate through several time steps, "unroll" the network, and then back propagate across the sequence of inputs.
Why would you not just ...
- 2,191
17
votes
2
answers
7k
views
What is the intuition behind a Long Short Term Memory (LSTM) recurrent neural network?
The idea behind Recurrent Neural Network (RNN) is clear to me. I understand it in the following way:We have a sequence of observations ($\vec o_1, \vec o_2, \dots, \vec o_n$) (or, in other words, ...
- 724
16
votes
2
answers
12k
views
Timeseries analysis procedure and methods using R
I am working on a small project where we are trying to predict the prices of commodities (Oil, Aluminium, Tin, etc.) for the next 6 months. I have 12 such variables to predict and I have data from Apr,...
16
votes
2
answers
8k
views
Getting started with neural networks for forecasting
I need some resources to get started on using neural networks for time series forecasting. I am wary of implementing some paper and then finding out that they have greatly over stated the potential of ...
user4280
11
votes
2
answers
18k
views
How can a network with only ReLU nodes output negative values?
I'm trying to use an api with a feedforward neural network for time series forecasting.
For dense aggregate data it works fine, but for sparse data it sometimes forecasts negative values, even though ...
- 12.1k
11
votes
3
answers
1k
views
Why convert spectrogram to RGB for machine learning?
I've seen a few publications that feed an RGB image of a spectrogram to a neural net, and someone claiming a network does better with RGB than grayscale or raw spectrogram.
A spectrogram is ...
10
votes
2
answers
7k
views
Best use of LSTM for within sequence event prediction
Assume the following 1 dimensional sequence:
A, B, C, Z, B, B, #, C, C, C, V, $, W, A, % ...
Letters A, B, C, .. here ...
- 201
9
votes
2
answers
35k
views
Example of time series prediction using neural networks in R
Anyone's got a quick short educational example how to use neural networks (nnet in R for example) for the purpose of prediction?
Here is an example, in R, of a ...
- 399
9
votes
3
answers
714
views
Time series forecasting: from ARIMA to LSTM
I am looking for resources on the techniques for time series forecasting. It seems that there are three approaches, listed below in the order of their machine learning-ness (and correspondingly their ...
- 4,487
9
votes
1
answer
468
views
What is a good approach to split 3 years of hourly data in a train, validation and test set for an electricity price forecasting neural network?
I would like to train a simple neural network to forecast electricity prices in a certain region. However, I only have a 'limited' amount of data available (3 sequential years of the historical price, ...
- 93
9
votes
2
answers
15k
views
Difference between Time delayed neural networks and Recurrent neural networks
I would like to use a Neural Network to predict financial time series. I come from an IT background and have some knowledge of Neural Networks and I have been reading about these:
TDNN
RNN
I have ...
- 93
9
votes
2
answers
911
views
Mathematical justification for using recurrent neural networks over feed-forward networks
I was wondering and trying to understand if there exists any mathematical reason behind the superiority of RNNs over Feed-forward networks when dealing with sequential data. For example when modeling ...
- 391
9
votes
1
answer
3k
views
RNN learning sine waves of different frequencies
As a warm up with recurrent neural networks, I'm trying to predict a sine wave from another sine wave of another frequency.
My model is a simple RNN, its forward pass can be expressed as follow:
$$
\...
- 203
9
votes
1
answer
149
views
Is there a ML or DL tool that can learn to detect periodically occurring patterns in a one dimensional time series?
I am trying to create a tool that labels refrigerator temperature readings. A reading is taken every 5 minutes, and its label identifies whether of not it was taken while the refrigerator was ...
- 91
8
votes
2
answers
16k
views
Classification in time series: SVMs, Neural Networks, Random Forests or non parametric models
My dataset is made of a label, $y_{t}$, which is the dependent variable, and about 20 columns of independent numeric variables, $X_{t}$, $t=1,2,...,T$.
These samples are time series and my goal is to ...
- 637
8
votes
3
answers
3k
views
k-fold CV of forecasting financial time series -- is performance on last fold more relevant?
I am working on an ANN-based forecasting model for a financial time series. I'm using 5-fold cross-validation and the average performance is so so. Performance on the last fold (the iteration where ...
- 83
8
votes
3
answers
8k
views
Recurrent neural networks in R
I've heard a bit about using neural networks to forecast time series, specifically recurrent neural networks.
I was wondering, is there a recurrent neural network package for R? I can't seem to find ...
- 24.4k
8
votes
1
answer
36k
views
Formula for number of weights in neural network
I'm trying to find a way to estimate the number of weights in a neural network. Let's look a simple example.
nnetar(1:10) (from the forecast package in R) gives me ...
- 125
8
votes
3
answers
2k
views
Multilayer neural networks for multivariate temporal data
I'm looking for a way to model and extract features from multivariate temporal data (e.g., multi-channel audio recordings).
I'm specifically interested in deep learning methods such as RBM, sparse ...
- 1,626
8
votes
2
answers
6k
views
What is a good way to test a simple Recurrent Neural Network
I have coded up a simple real-value regression RNN in theano.
What kind of dataset should I test it on?
How should I go about testing it?
My structure is:
Univariate (for now) timeseries, $x_{in}(...
8
votes
1
answer
4k
views
7
votes
2
answers
11k
views
Time series prediction: Neural Network (nnetar) vs. exponential smoothing (ets)
When I make a forecast for the univariate time series $x_1=1, x_2=2, \dots, x_{14} = 14$, why does the nnetar() function in R (which uses a neural network) not ...
- 255
7
votes
1
answer
10k
views
Why CNN is suitable for time-series data?
I am confused by the statements that I came across in two different papers.
The statement from the paper titled as "Detecting Cyber Attacks in Industrial Control Systems Using Convolutional ...
- 325
7
votes
2
answers
3k
views
Implementing Neural Network for time series
I am currently working on neural networks for time series forecasting. My doubt is: do we need to take into account issues like trend, non-stationarity and seasonality while using neural networks ...
- 1,556
7
votes
1
answer
23k
views
Optimum number of epochs and neurons for an LSTM network
I wanted to know if there's a way to select an optimum number of epochs and neurons to forecast a certain time series using LSTM, the motive being automation of the forecasting problem, i.e. the ...
- 71
7
votes
2
answers
7k
views
Predicting time series with NNs: should the data set be shuffled?
Suppose I'm trying to predict time series with a neural network. The data set is created from a single column of temporal data, where the inputs of each pattern are ...
- 708
7
votes
2
answers
156
views
How to set up a DL classification model so that it selects from an ever changing menu
The question is edited for clarity after tchainzzz's comments about meta-learning.
Let's say we have 10,000 pet pictures and 10,000 kids. Each kid is presented with 10 randomly picked pet pictures at ...
- 205
7
votes
1
answer
10k
views
How to forecast multivariate time-series 'accurately' with a large number of unknown factors using R?
I am relatively new to statistics and not formally trained but have been given a complex problem to solve and need some guidance. I realise that I am out of my depth a bit here but would appreciate ...
- 111
7
votes
1
answer
3k
views
Combine several softmax output probabilities
I would like to combine the outputs of five neural networks, each with a softmax output layer of three classes each. A typical, example output is shown below:-
where Figure 1 is the output of model 1,...
- 4,869
6
votes
4
answers
609
views
Deep Learning based time series forecasting
According to the paper "Statistical and Machine Learning forecasting methods: Concerns and ways forward", it looks like the recent DNN-based approach has weaker predictive power in extrapolation, i.e. ...
6
votes
4
answers
7k
views
Time series analysis with neural networks
I'm new to neural networks and machine learning and I was wondering how you use time series data to set the weights of a regular FNN, and how you use the ending weights to forecast the time series. In ...
- 61
6
votes
1
answer
4k
views
Role of delays in LSTM networks
LSTM network is assumed to be about memory, keeping the important information for predictions.
If it is the case, why do we need to consider delayed inputs as well?
My assumption would be that the ...
- 2,846
6
votes
1
answer
536
views
Recurrent networks mimicking previous / current input
So I have been trying to train an LSTM to predict the values of a certain stock. The error was pretty low, so I decided to create a graph out of the test set. It looked like this:
Red: actual, Black: ...
- 1,314
6
votes
1
answer
3k
views
How to choose between plain vanilla RNN and LSTM RNN when modelling a time series?
What are the criteria used to choose between plain vanilla RNN and LSTM RNN when you have to model a generic time series?
- 949
6
votes
1
answer
3k
views
What is the most efficient method to handle long time sequences (LSTM)?
I am using LSTM and I have several long time sequences of varying length. Most of them are about 6,000-7,000 timesteps on average, but several are around 40,000 long. I am not sure which of this would ...
- 541
6
votes
1
answer
5k
views
Online learning in LSTM
Recently, I have been working on RNNs (LSTM specifically) to do time series prediction and I have used different frameworks such as deeplearning4j and theano (keras). As you may know, one of the ...
- 366
5
votes
1
answer
893
views
How is an RNN (or any neural network) a parametric model?
I'm going through this paper A Multi-Horizon Quantile Recurrent Forecaster. The authors state that:
3.1. Loss Function
In Quantile Regression, models are trained to minimize the total ...
- 12.1k
5
votes
1
answer
3k
views
Cannot understand LSTM inference
I seem to have stumbled on a hole in my understanding around LSTMs. In short, I cannot understand how even a simple one is actually fed samples, upon inference time/training time. Here are the details:...
- 1,685