3
$\begingroup$

I implemented my own NN framework and did a regression on a small dataset that follows pretty good the target only is delayed forward, by 1 time point. This is how the validation data looks: enter image description here

The curios thing is that the training data looks the same:

enter image description here

Although the prediction has almost the same shape as the target signal, the 1 time point difference creates a significant difference between the 2 signals.

Did anyone encountered this kind of problem and know how to solve it ? For other datasets the regression works very well, also on timeseries, also for classification, using my NN framework. It could be a bug, but why does it work very well on MNIST10 ? and other timeseries datasets ?

I used a classic RNN, not a LTSM. Input data is a time lagged input from (t, t-1, ....., t-12). Input data was normalized in [0,1], then sent to hidden layer 1 activated with tanh, then to hidden layer 2 activated with tanh, and then to error layer activated with sigmoid(tried also linear, same thing). Hidden layers each have a context layer that stores the activation of the hidden layer at the previous time t-1 to feed it at time t to the hidden layer.

I sent batches of 128 sequences of length 10,13,30,40 and other. The train dataset has about 2950 time points, and the val dataset has 155 time points.

The accuracy is: TRAIN Root mean squared error: 0.038273 TRAIN Correlation coefficient: 0.974084 VAL Root mean squared error: 0.054285 VAL Correlation coefficient: 0.861191

Does anyone know how this problem can be solved using RNNs ? I am not interested in using other statistical models or Google frameworks.

Many thanks, Viorel

Improve this question
$\endgroup$
4
  • $\begingroup$ I think I understand the problem as explained here stats.stackexchange.com/questions/197494/… $\endgroup$
    – Chelaru Viorel
    Commented May 5, 2016 at 9:54
  • $\begingroup$ I also simulated a random walk starting from a signal S and created R = S + small noise and indeed R follows very wel S but with a 1 time delay. Ok, I now understand the problem, my RNN takes any sequence and outputs the last time point in the sequence + some noise. $\endgroup$
    – Chelaru Viorel
    Commented May 5, 2016 at 10:05
  • $\begingroup$ But why does this happen ? All I see that is strange, is that the trained weights from the context layer to the hidden layer are very small, in interval [-0.01, 0.01] when the other trained weights stay in [-0.5, 0.5]. Maybe this is the cause ? $\endgroup$
    – Chelaru Viorel
    Commented May 5, 2016 at 10:05
  • $\begingroup$ I sort of fixed this. I filtered the signal with a gaussian, and the neural net successfully predicted the smoothed signal. But then I have to add back the noise to the prediction. $\endgroup$
    – Chelaru Viorel
    Commented Jun 13, 2016 at 17:47

1 Answer 1

Reset to default
4
$\begingroup$

This is not surprising and happens often with much simpler linear time series models. Imagine you have a martingale process (a process that basically moves randomly in every step). The optimal forecast is whatever value you had in the previous period.

What I think your graph is telling you is that, according to your model, its best forecast is whatever it saw last.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.