2
$\begingroup$

I am having an issue using neural networks to predict time series. Some predicted data fits with the expected data, as bellow: (In black the real time series and in blue the output of my neural network)

Australia energy demand Time serie: Australia energy demand.

But with the same code, with other time series, the predicted data does not fits with the expected data, and has a delay of one unit, as bellow:

enter image description here Time serie: Walmart Stock price. enter image description here Time serie: Dollar libra exchange.

I found some articles about some variations of neural networks and at the results section shows the plot with the delay like my results, as bellow:

enter image description here Time serie: Dollar libra exchange. (Article link: http://www.sciencedirect.com/science/article/pii/S1877050915015793)

Anyone knows if this is a common behavior or can be something wrong with my code ? I am having this issue about three months ago, and since there I am trying to figure out some bug in my code but is all right.

Thanks and I appreciate any tip.

Improve this question
$\endgroup$
2
  • $\begingroup$ What type of neural network are you using and what are your parameters? $\endgroup$
    – noumenal
    Commented Feb 19, 2016 at 20:23
  • $\begingroup$ @noumenal I am using the Extreme Learning Machine. Source code ntu.edu.sg/home/egbhuang/elm_codes.html. Each data set has different parameters. $\endgroup$
    – ViniciusArruda
    Commented Feb 22, 2016 at 13:26

3 Answers 3

Reset to default
2
$\begingroup$

Something could always be wrong with your code, but this type of behavior would be consistent with a model that was similar to a random walk.

A random walk (without drift) would be specified as follows; $$ y_t = y_{t-1} + \varepsilon_t $$ where $E[e_t] = 0$. Thus $E[Y_t|y_1,...,y_{t-1}] = y_{t-1}$. So when you graph your predictions against actual observations, it will look as though your predictions are delayed by one unit, when in fact the forecasts are just using the contemporaneous observation as next periods prediction.

Not knowing anything else about your neural network model and code, my best guess would be that the model converged on something similar to a random walk (the graph does suggest that there are additional trend/drift components at work too though).

Improve this answer
$\endgroup$
3
  • $\begingroup$ Sorry for the delay in answering you, but I was researching about my neural network and the random walk. I have concluded that my model, for some time series, is having this issue. Can you give me some reference, if you have, about it and how to get rid of this problem in neural networks or other kind of modeling ? The unique solution for me was tune the parameters from my net, but this not helped a lot. $\endgroup$
    – ViniciusArruda
    Commented Apr 19, 2016 at 19:42
  • $\begingroup$ @ViniciusArruda I don't know if I would call this instance a "problem". It doesn't look great on a plot, but forecasting next period's value with that of the current period is sometimes one of the best predictions available. This is especially true with financial time series that do not have strong seasonal patterns. I do not have a citation for you, but modeling some equities and stock market indices as random walks over long run periods is common place and there are theoretical arguments for this behavior as well. $\endgroup$
    – Zachary Blumenfeld
    Commented Apr 20, 2016 at 18:49
  • $\begingroup$ Sorry for lack of citation but a google search into random walks in finance should get you started. If you have not done so already, you could try transforming the series to percentage changes so that it is stationary. I am not familiar enough with neural networks to know whether or not they can deal with non-stationary. My brief exposure to the basics suggests that they cannot do so directly, at least in the standard case. By transforming the series you can perhaps leverage the neural network to find non-linear relationships in percentage changes that are not apparent between price levels. $\endgroup$
    – Zachary Blumenfeld
    Commented Apr 20, 2016 at 19:02
0
$\begingroup$

The problem is because of the existing overlap between the last record of train data and the first record of test data. You should simply drop the last time-stamp of train data!

train = train.iloc[:-1, :]
Improve this answer
$\endgroup$
0
$\begingroup$

Usually, we need to compensate the prediction horizon to make the model casual. Example, if have some model like this:

y[k] = *y[k - 10] + *u...

where y is the measurement and u is some input.

In this case, we need to know data from [k - 10], that we dont have. One solution that I usually do is to repeat the first data for the prediction horizon (10x in this case) to make the model causal.

Improve this answer
$\endgroup$
1
  • $\begingroup$ how do you "repeat the data"? $\endgroup$
    – utobi
    Commented Nov 29, 2023 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.