1
$\begingroup$

Let say I simulate an AR(1) process. We can easily model the data with a vanilla NN with a single neuron as it is about finding the linear relationship between $y$ and $y_{t-1}$.

Now, how about a MA(1)? Can a vanilla NN model that MA process? And if not why (w.r.t universal approximation theorem) and what kind of structure would you need?

Improve this question
$\endgroup$
5
  • $\begingroup$ With an MA(1), there's a linear relationship between $y_t$ and the previous error term so I imagine you can do a similar thing except that the coefficient is the coefficient of the previous error, $\epsilon_{t-1}$. Note that MA(1) is $y_t = \theta \times \epsilon_{t-1} + \epsilon_t$. $\endgroup$
    – mlofton
    Commented Jun 7, 2021 at 21:56
  • $\begingroup$ If you give $\epsilon_{t-1}$ as feature as well, why not. $\endgroup$
    – gunes
    Commented Jun 8, 2021 at 8:06
  • $\begingroup$ The thing is that errors terms are not observed values. $\endgroup$
    – Junior Hechinger
    Commented Jun 8, 2021 at 10:53
  • $\begingroup$ You could use a simple one node NN to get those error terms and then feed those into more layers to provide the actual forecast. So like a 2 stage NN. $\endgroup$
    – Tylerr
    Commented Jun 9, 2021 at 14:12
  • $\begingroup$ @Junior Hechinger: The estimates of error terms are available. $\epsilon_{t-1}$ is the just the error from the previous prediction,. So, $\hat{\epsilon}_{t-1} = \hat{y}_{t-1} - \theta \hat{\epsilon}_{t-2}$ and so on and so forth. $\endgroup$
    – mlofton
    Commented Jun 12, 2021 at 18:51

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.