Skip to main content
27 votes

Confused about Autoregressive AR(1) process

You have two problems--and one of them is interesting. Without a noise term, the series is no longer stationary. Its value is increasing asymptotically, but definitely, toward $1:$ ARIMA applies ...
whuber's user avatar
  • 325k
25 votes

How does ACF & PACF identify the order of MA and AR terms?

The quotes are from the link in the OP: Identification of an AR model is often best done with the PACF. For an AR model, the theoretical PACF “shuts off” past the order of the model. The phrase “...
Antoni Parellada's user avatar
24 votes
Accepted

Why do we care if an MA process is invertible?

Invertibility is not really a big deal because almost any Gaussian, non-invertible MA$(q)$ model can be changed to an invertible MA$(q)$ model representing the same process by changing the parameter ...
Jarle Tufto's user avatar
  • 11.2k
17 votes
Accepted

Why is OLS estimator of AR(1) coefficient biased?

As essentially discussed in the comments, unbiasedness is a finite sample property, and if it held it would be expressed as $$E (\hat \beta ) = \beta$$ (where the expected value is the first moment ...
Alecos Papadopoulos's user avatar
17 votes

If an auto-regressive time series model is non-linear, does it still require stationarity?

If the purpose of your model is prediction and forecasting, then the short answer is YES, but the stationarity doesn't need to be on levels. I'll explain. If you boil down forecasting to its most ...
Aksakal's user avatar
  • 61.5k
14 votes
Accepted

What is the difference between deterministic and stochastic model?

The video is talking about deterministic vs. stochastic trends, not models. The highlight is very important. Both your models are stochastic, however, in the model 1 the trend is deterministic. The ...
Aksakal's user avatar
  • 61.5k
13 votes

What is the difference between deterministic and stochastic model?

As Aksakal mentioned in his answer, the video Ken T linked describes properties of trends, not of models directly, presumably as part of teaching about the related topic of trend- and difference-...
ido's user avatar
  • 231
13 votes

Why is the dickey fuller test different from a simple t-test

You are right that the test statistic is just a standard t-statistic. It, however, follows a different null distribution, i.e., using critical values from the t or normal distribution would lead to ...
Christoph Hanck's user avatar
11 votes
Accepted

How to write variance covariance matrix of AR(1) process in R

The covariance between an observation at time $t_i$ and time $t_j$ is $$ \frac{\sigma^2}{1-\phi^2}\phi^{|t_i-t_j|} $$ If the delta's are time gaps between the time ...
Jarle Tufto's user avatar
  • 11.2k
11 votes
Accepted

Where is $|\theta|<1$ used in recursive method derivation of invertibility of MA(1)?

It's not a silly question --- this is a common misconception in the recursive method, and I've seen many people make the same mistake. The problem here is that your "and so on" glosses over ...
Ben's user avatar
  • 126k
10 votes
Accepted

Why are exponential smoothing models not considered auto-regressive?

For an autoregressive model, non-linear or linear, the number of lags must be finite. An ETS(A,N,N) model can be written as an AR($\infty$) model, but not as an autoregressive model with finite lags. ...
Rob Hyndman's user avatar
10 votes
Accepted

Exponential decay of ACF of AR(p) process

$f(h) =\phi^h$ is an exponential function.
markowitz's user avatar
  • 5,689
9 votes

Under what circumstances is an MA process or AR process appropriate?

I can provide what I think is a compelling answer to the first part of the question ("whence MA?") but am presently pondering an equally compelling answer to the second part of the question ("whence ...
Student's user avatar
  • 139
9 votes

Why is OLS estimator of AR(1) coefficient biased?

@Alecos nicely explains why a correct plim and unbiasedbess are not the same. As for the underlying reason why the estimator is not unbiased, recall that unbiasedness of an estimator requires that all ...
Christoph Hanck's user avatar
9 votes
Accepted

Random Forest regression for time series prediction

How is this different from utilizing 'later' data in the time series as testing? The approach you quote is called "rolling origin" forecasting: the origin from which we forecast out is "rolled ...
Stephan Kolassa's user avatar
9 votes

Simulate AR(1) process in R with specified nonzero mean and AR coefficient

Notice that this is the same model as $$ X_t - 10 = .5(X_{t-1} - 10) + Z_t . $$ $10$ is the mean, while $5$ was the intercept. This means we can add ten to the mean zero series. In other words, if ...
Taylor's user avatar
  • 20.8k
9 votes
Accepted

The explosive AR(1) process with $\varphi>1$, where was this first represented as a stationary, but non-causal, time-series?

The question suggests some basic confusion between the equation and the solution The Equation Let ${\varphi} > 1$. Consider the following (infinite) system of equations---one equation for each $t\...
Michael's user avatar
  • 3,328
9 votes
Accepted

Asymptotic Distribution of Explosive AR(1) Process

This is ordinary algebra: the fact that the $X_t$ and $\epsilon_t$ are random variables is immaterial. You only need $X_0=0$ (and no restriction on $\rho$) to carry out these simple steps based on ...
whuber's user avatar
  • 325k
8 votes

Intuition behind the characteristic equation of an AR or MA process

When trying to get an intuitive understanding of formal mathematical models, it is usually best to start with a simple model and then generalise later. So, with that in mind, let's start with an AR$(...
Ben's user avatar
  • 126k
8 votes

Generate AR(1) process with different $y_0$ values in R

arima.sim does not allow such fine control. But if you look at the function definition, you will notice that it is just applying ...
Rob Hyndman's user avatar
8 votes
Accepted

Finding variance of AR process

$$\text{Var}(y_t)=\text{Var}(\phi y_{t-1}) + \text{Var}(\varepsilon_{t}).$$ As we know, $E(\varepsilon_{t}^2)=\sigma^2$. Then we have: $$\text{Var}(y_t)=\text{Var}(\phi y_{t-1}) + \sigma^2.$$ Now ...
adrian1121's user avatar
  • 1,116
8 votes

How does ACF & PACF identify the order of MA and AR terms?

Robert Nau from Duke's Fuqua School of Business gives a detailed and somewhat intuitive explanation of how ACF and PACF plots can be used to choose AR and MA orders here and here. I give a brief ...
Lino Ferreira's user avatar
8 votes
Accepted

Difference between MA and AR

A key difference which I failed to appreciate: the MA model predictions of $x_t$ include $\epsilon_{t-1}$ in its computation whereas the AR model only predicts based on $x_{t-1}$ without (explicit) ...
Jonas Lindeløv's user avatar
8 votes
Accepted

Why this OLS fitting will converge to (0,-0.5)?

This is a law of large numbers in action. Let $\rho$ be the parameter (the lag-1 correlation) and let $\varepsilon_i$ be a sequence of iid standard Normal variables, so that for $i=1, 2, \ldots,$ the ...
whuber's user avatar
  • 325k
8 votes
Accepted

Expectation of the ratio of sum (XY) and sum(X)

I will assume $a=0$ and $b=1$ in the following. Here is a simulation experiment to look at the variability of the expectation in $M$: ...
Xi'an's user avatar
  • 106k
8 votes

AR Forecasting one day ahead

+1 to Richard's answer. A few additions: First off, as Richard notes, you are using the full training set to estimate the autoregression coefficients, which is what ...
Stephan Kolassa's user avatar
8 votes
Accepted

A question reagarding stationarity of an AR(2) model

This is a consequence of the (general) fact that (auto)correlation matrices like $$ \begin{pmatrix} 1&\rho_1&\rho_2\\ \rho_1&1&\rho_1\\ \rho_2&\rho_1&1\\ \end{pmatrix} $$ are ...
Christoph Hanck's user avatar
7 votes
Accepted

How many lags should I include in time series prediction?

Looking at individual autocorrelations may help in simple cases, but this way you could miss lags that are important only jointly but not individually. Alternatively, you may try the following: ...
Richard Hardy's user avatar
7 votes

Why is OLS estimator of AR(1) coefficient biased?

Expanding on two good answers. Write down the OLS estimator: $$\hat\beta =\beta + \frac{\sum_{t=2}^Ty_{t-1}\varepsilon_t}{\sum_{t=2}^Ty_{t-1}^2}$$ For unbiasedness we need $$E\left[\frac{\sum_{t=2}^...
mpiktas's user avatar
  • 35.1k
7 votes

Reproducing sinusoid with autoregressive discrete model

You can prove that the coefficients of a AR(2) model without added noise can reproduce a sine wave. You can write your AR model as: x(n+1) = a1*x(n) + a2*x(n-1) ...
flavio martinelli's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible