In Wikipedia, an autoregressive model is defined in terms of an AR(p) linear process as

The autoregressive model specifies that the output variable depends linearly on its own previous values and on a stochastic term (an imperfectly predictable term); thus the model is in the form of a stochastic difference equation. In machine learning, an autoregressive model learns from a series of timed steps and takes measurements from previous actions as inputs for a regression model, in order to predict the value of the next time step

I see how AR is used with respect to something with normal innovations like ARMA. However, how is any machine learning algorithm linear? However, the term "autoregressive model" and "autoregression" is used commonly on the internet with respect to any model that has features composed of the past values of the time series, linear or non-linear.

Suppose we have an MLP. Then the value $y_t$ of time series does depend on values $y_{t-1}..y_{t-p}$, however, this dependence is nonlinear due to activations.

Or if we take GLM with non-normal error. Can it be considered autoregressive process since the value $y_t$ depends non-linearly on its previous terms (due to some link function).

Same with SVM, Random Forests, anything else used in ML. Can someone explain what is meant by autoregressive model, if there is a mistake in Wiki, or if there are multiple interpretations of the term "autoregressive model" in ML vs statistics? Is the term use the same with respect to the modelling, and the stochastic processes too?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.