15

As you noted, (1) an AR model relates the value of an observation $x$ at time $t$ to the previous values, with some error: $$ x_t = \phi x_{t-1} + \varepsilon_t $$ Let's substitute in $ x_{t-1} $, and then $ x_{t-2} $: $$\begin{aligned} x_t &= \phi (\phi x_{t-2} + \varepsilon_{t-1}) + \varepsilon_t \\ &= \phi^2x_{t-2} + \phi\varepsilon_{t-1} + \...


14

If the purpose of your model is prediction and forecasting, then the short answer is YES, but the stationarity doesn't need to be on levels. I'll explain. If you boil down forecasting to its most basic form, it's going to be extraction of the invariant. Consider this: you cannot forecast what's changing. If I tell you tomorrow is going to be different than ...


12

We estimate by OLS the model $$x_{t} = \rho x_{t-1} + u_t,\;\; E(u_t \mid \{x_{t-1}, x_{t-2},...\}) =0,\;x_0 =0$$ For a sample of size T, the estimator is $$\hat \rho = \frac {\sum_{t=1}^T x_{t}x_{t-1}}{\sum_{t=1}^T x_{t-1}^2} = \rho + \frac {\sum_{t=1}^T u_tx_{t-1}}{\sum_{t=1}^T x_{t-1}^2}$$ If the true data generating mechanism is a pure random walk, ...


11

VAR is actualy an equation. We say that process $X_t$ is VAR when it satisfies the following equation: $$X_t=\alpha+\Phi_1X_{t-1}+...+\Phi_pX_{t-p}+\varepsilon_t,$$ where $\Phi_i$ are matrices and $\varepsilon_t$ is white noise process. If the process satistfying this equation is stationary we say that the VAR is stationary. Given matrices $\Phi_i$ you can ...


11

For a second-order stationary series it is the correlation coefficient between the dependent value and its lag. Specify $$y_{t+1} = \beta y_t + u_{t+1}\qquad u_{t+1}= \text{white noise}$$ The correlation coefficient between $y_{t+1}$ and $y_{t}$ is defined as usual $$\rho_{(1)} = \frac{\text{Cov}(y_{t+1},y_{t})}{\sigma(y_{t+1})\sigma(y_t)}$$ Now $$\text{...


11

As Aksakal mentioned in his answer, the video Ken T linked describes properties of trends, not of models directly, presumably as part of teaching about the related topic of trend- and difference-stationarity in econometrics. Since in your question, you asked about models, here it is in the context of models: A model or process is stochastic if it has ...


11

The video is talking about deterministic vs. stochastic trends, not models. The highlight is very important. Both your models are stochastic, however, in the model 1 the trend is deterministic. The model 2 doesn't have a trend. Your question text is incorrect. The model 2 in your question is AR(1) without a constant, while in the video the model is a ...


10

One important and useful result is the Wold representation theorem (sometimes called the Wold decomposition), which says that every covariance-stationary time series $Y_{t}$ can be written as the sum of two time series, one deterministic and one stochastic. $Y_t=\mu_t+\sum_{j=0}^\infty b_j \varepsilon_{t-j}\,$, where $\mu_t$ is deterministic. The second ...


10

You are right that the test statistic is just a standard t-statistic. It, however, follows a different null distribution, i.e., using critical values from the t or normal distribution would lead to tests that would not reject in $\alpha$% of the cases when the null is true. See Estimation of unit-root AR(1) model with OLS for an assumption that is ...


9

Let $X_t$ be a zero-mean covariance-stationary time series such that $$X_t = \varphi_1 X_{t-1} + \varphi_2 X_{t-2} + \varepsilon_t$$ where $\varepsilon_t$ is white noise. Using $L$ to mean the lag (backshift) operator, the above can be expressed as $$(1-\varphi_1L - \varphi_2L^2)X_t=\varepsilon_t . \tag{1}$$ Since $X_t$ is a covariance-stationary AR(2) ...


9

The models with lagged independent variables are called distributed lag models. Usually introductory econometrics texts have a section or chapter dedicated to them. They were more popular in the 1980s, and actually Christopher Sims got his Nobel prize in economics for the work on such models (see the article Money, Income and Causality). Nowadays they are ...


9

$$y_t = \alpha + \beta t$$ and $$y_t = y_{t-1} + \beta$$ are the same up to a constant: take the second equation and iteratively substitute for $y_{t-1}$ to get $$y_t = y_{t-1} + \beta = y_{t-2} + 2 \beta = \dotsc = y_0 + \beta t.$$ So if $\alpha = y_0$, the two coincide. If not, the difference is $\alpha - y_0$, which is a constant. Meanwhile, $$...


9

As essentially discussed in the comments, unbiasedness is a finite sample property, and if it held it would be expressed as $$E (\hat \beta ) = \beta$$ (where the expected value is the first moment of the finite-sample distribution) while consistency is an asymptotic property expressed as $$\text{plim} \hat \beta = \beta$$ The OP shows that even though ...


8

If you fit an arima model with external regressions, you MUST provide newxreg to the predictions function. This goes for arima, auto.arima, and Arima. You cannot provide external regressors to ets. xreg should contain the exogenous variables for the training set, and newxreg should contain those variables for the prediction set. If the nexreg contains ...


8

Auto-regressive models (ARIMA) use previous values as predictors depending upon the form of the model and forecasts are adaptive in form generally responding to previous values. Models using time as a predictor can be understood as using previous values to estimate the model parameters (thus previous values do come into play ) but they are otherwise not ...


8

The covariance between an observation at time $t_i$ and time $t_j$ is $$ \frac{\sigma^2}{1-\phi^2}\phi^{|t_i-t_j|} $$ If the delta's are time gaps between the time points $t_1,t_2,\dots,t_5$, then each $t_i$ are given by cumulative sums of the delta's. In R do delta <- c(0,1,3,4,8) phi <- .5 sigma <- 1 t <- cumsum(delta) Sigma <- sigma^2/(1-...


7

The quotes are from the link in the OP: Identification of an AR model is often best done with the PACF. For an AR model, the theoretical PACF “shuts off” past the order of the model. The phrase “shuts off” means that in theory the partial autocorrelations are equal to 0 beyond that point. Put another way, the number of non-zero partial ...


7

Invertibility is not really a big deal because almost any Gaussian, non-invertible MA$(q)$ model can be changed to an invertible MA$(q)$ model representing the same process by changing the parameter values. This is mentioned in most textbooks for the MA(1) model but it is true more generally. As an example, consider the MA(2) model $$ z_t = (1-0.2B)(1-2B)...


7

For an autoregressive model, non-linear or linear, the number of lags must be finite. An ETS(A,N,N) model can be written as an AR($\infty$) model, but not as an autoregressive model with finite lags. A few other exponential smoothing models can be written in AR($\infty$) form, but none can be written as an autoregressive model with finite lags. See https://...


6

ARIMA is more general. It allows fitting certain nonstationary time series and even stationary series that cannot be fit by low order autoregressive models.


6

Let us rewrite $x_t, x_{t-1}, \dots, x_{t-K+1}$ in terms of $x_{t-K}$ $$x_t=c\left(1+\varphi+\dots+\varphi^{K-1}\right)+\varepsilon_t+\varphi\varepsilon_{t-1}+\dots+\varphi^{K-1}\varepsilon_{t-K+1}+\varphi^Kx_{t-K}$$ $$x_{t-1}=c\left(1+\varphi+\dots+\varphi^{K-2}\right)+\varepsilon_{t-1}+\varphi\varepsilon_{t-2}+\dots+\varphi^{K-2}\varepsilon_{t-K+1}+\...


6

You should use the following formula: Y(t) = 0.3793*(Y(t-1) - 9132.46 - 22.0469*X(t-1)) + 9132.46 + 22.0469*X(t). Example to replicate out-of-sample forecasts: require(forecast) set.seed(123) n <- 100 xreg <- rnorm(n) x <- arima.sim(n=n, model=list(ar=0.4)) + 2 + 0.8 * xreg fit <- arima(x, order=c(1,0,0), include.mean=TRUE, xreg = xreg) ...


6

First, you should decide on using a univariate or a multivariate model. It seems reasonable to think that oil price and unemployment are causal for the air travel demand and not the other way around. Thus, in line with one of the answers to this post, you may address your study in a univariate setting. If the previous assumption is not appropriate, then you ...


6

This is not really an answer but too long for a comment, so I post this anyway. I was able to get a coefficient greater than 1 two times out of a hundred for a sample size of 100 (using "R"): N=100 # number of trials T=100 # length of time series coef=c() for(i in 1:N){ set.seed(i) x=rnorm(T) # generate T ...


6

Its not common to give away the answer for a question but I had this code ready so I can just as well post it together with an explanation of what each line does. This code does however not discard the first 100 observations so that is something you can do on your own. %Simulate AR(3) T = 1000; %Set how many observations you need ...


6

Let us write the joint density as \begin{equation} p(y_1,\ldots,y_T) = p(y_1)\,p(y_2\mid y_1) \, p(y_3 \mid y_2,y_1) \ldots \, p(y_T \mid y_{T-1},\ldots y_1). \end{equation} Furthermore, since the process is AR(1), the past values influence future values only via the latest value, i.e., we have the Markov property $p(y_t \mid y_1,\ldots,y_{t-1}) = p(y_t \mid ...


6

My response is more from a practical perspective. I'm specifically going to address your second part of question: why can't we use machine learning techniques for time series? Reason #1: there is NO empirical evidence that machine learning are known to be superior than simple statistical time series models. Why bother with machine learning when there is no ...


6

@Alecos nicely explains why a correct plim and unbiasedbess are not the same. As for the underlying reason why the estimator is not unbiased, recall that unbiasedness of an estimator requires that all error terms are mean independent of all regressor values, $E(\epsilon|X)=0$. In the present case, the regressor matrix consists of the values $y_1,\ldots,y_{...


6

How is this different from utilizing 'later' data in the time series as testing? The approach you quote is called "rolling origin" forecasting: the origin from which we forecast out is "rolled forward", and the training data is updated with the newly available information. The simpler approach is "single origin forecasting", where we pick a single origin. ...


6

A stable filter is a filter which exists, and is causal. Causal means that your current observation is a function of past or contemporaneous noise, not future noise. Why do they use the word stable? Well, intuitively, you can see what happens when you simulate data from the model if $|\phi| > 1$. You will see the process could not hover around some mean ...


Only top voted, non community-wiki answers of a minimum length are eligible