0
$\begingroup$

I’ve a question related to the NARX architecture of neural nets:

What is the usage of the tapped delay line in this architecture?

My problem is, that I can’t figure out the difference of this compared to a basic multilayer perceptron. Let’s say I have a set of data $S$ which represents a time series. I always want to predict the $k_{th}$ values of this series by training the network on subsets of $S$ from $j$ to $k-1$. So, the amount of tapped delay lines of the network is equal to the size of values of the subsets of $S$. On predictions of the $k_{th}$ value, I backpropage the error between the expected and the predicted value to adjust the weights. For the next prediction, I start with the expected kth value to be the last in the subset. So what I actually do is to use a sliding window over my training data where the expected, the target value is always feed back into the net with when the sliding window moves.

Can’t I just use a MLP for this? Is it not just a matter of the training procedure?

$\endgroup$

1 Answer 1

4
$\begingroup$

First of all the use of the tap delay line in this architecture is to express an input vector that is composed of the time-series data of one recorded variable.

So if you have N number of data in a set S, consider t to be an index of a given data-point at time t. Now if your defined delay is d, meaning how many time-steps in the past you are incorpoparting in the tap delay (in other words how far back in the past you want to look) then your Tap Delay Line(TDL) is:

TDL=[s(t), s(t-1), s(t-2)..., s(t-d)]

Now off course d < N otherwise you would not be able to build a TDL as there will be not enough data in the past. If now you want to train your network the target of this input vector would be the time-step:

target = s(t+1)

If you are predicting then the prediction is pushed into the TDL like so:

TDL=[s(t+1), s(t), s(t-1)..., s(t-(d-1))]

Example

Imagine you have time-series data set S from a sensor.

S = [0.3, 0.5, 0.7, 0.9, 0.11, 0.13] and delay d = 2. Then if we want to create a TDL for s(t) = 0.11 (t=4, t being the index of the array) the result is:

TDL = [0.11, 0.9, 0.7] and the target is target=[0.13].

Mathematicaly the NARX architecture is based on Takens's theorem.

In theory any regression model can implement a NARX architecture but MLPs are the most common. So to answer your question you can use an MLP to implement a NARX.

This paper is very useful if you want more detail on NARX.

Hope this helps!

$\endgroup$
2
  • $\begingroup$ Thank you very much much for your detailed answer! It's basically a reverse sliding window over a set of time series data, right? $\endgroup$
    – Bastian
    Commented Mar 9, 2018 at 18:08
  • $\begingroup$ I am not 100% sure what you mean by reverse window sliding. I think of it as rolling the time-series array to the right and then concatenating it to the original array vertically given that you have the array of data expanding horizontaly (like the example above). You can for example use numpy.roll. $\endgroup$
    – efkag
    Commented Mar 9, 2018 at 18:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.