We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [autoregressive]

The autoregressive (AR) model is a stochastic process modelling time series, which specifies the value of the series linearly in terms of the previous values.

Filter by
Sorted by
Tagged with
25
votes
1answer
14k views

How to understand SARIMAX intuitively?

I'm trying to understand a paper about electric load forecasting but I'm struggling with the concepts inside, specially the SARIMAX model. This model is used to the predict the load and uses many ...
21
votes
3answers
29k views

Under what circumstances is an MA process or AR process appropriate?

I understand that if a process depends on previous values of itself, then it is an AR process. If it depends on previous errors, then it is an MA process. When would one of either of these two ...
16
votes
2answers
965 views

If an auto-regressive time series model is non-linear, does it still require stationarity?

Thinking about using recurrent neural networks for time series forecasting. They basically implement a sort of generalized non-linear auto-regression, compared to ARMA and ARIMA models which use ...
13
votes
2answers
1k views

Why do we care if an MA process is invertible?

I am having trouble understanding why we care if an MA process is invertible or not. Please correct me if I'm wrong, but I can understand why we care whether or not an AR process is causal, ie if we ...
13
votes
2answers
1k views

AR(1) process with heteroscedastic measurement errors

1. The problem I have some measurements of a variable $y_t$, where $t=1,2,..,n$, for which I have a distribution $f_{y_t}(y_t)$ obtained via MCMC, which for simplicity I'll assume is a gaussian of ...
11
votes
3answers
14k views

How does ACF & PACF identify the order of MA and AR terms?

It's been more than 2 years that I am working on different time series. I have read on many articles that ACF is used to identify order of MA term, and PACF for AR. There is a thumb rule that for MA, ...
10
votes
3answers
10k views

What is the difference between deterministic and stochastic model?

Simple Linear Model: $x=\alpha t + \epsilon_t$ where $\epsilon_t$ ~iid $N(0,\sigma^2)$ with $E(x) = \alpha t$ and $Var(x)=\sigma^2$ AR(1): $X_t =\alpha X_{t-1} + \epsilon_t$ where $\epsilon_t$ ~...
10
votes
2answers
5k views

Random walk estimation with AR(1)

When I estimate a random walk with an AR(1), the coefficient is very close to 1 but always less. What is the math reason that the coefficient is not greater than one?
10
votes
1answer
10k views

Random Forest regression for time series prediction

I'm attempting to utilise RF regression to make predictions on the performance of a paper mill. I have minute by minute data for the inputs (rate and amount of wood pulp going in etc...) as well as ...
10
votes
1answer
1k views

Modelling auto-correlated binary time series

What are the usual approach to modelling binary time series? Is there a paper or a text book where this is treated? I think of a binary process with strong auto-correlation. Something like the sign of ...
10
votes
1answer
5k views

R and EViews differences in AR(1) estimates

The main problem is: I cannot obtain similar parameter estimates with EViews and R. For reasons I do not know myself, I need to estimate parameters for certain data using EViews. This is done by ...
10
votes
2answers
989 views

Unbiased estimator for AR($p$) model

Consider an AR($p$) model (assuming zero mean for simplicity): $$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$ The OLS estimator (equivalent to the conditional maximum ...
9
votes
3answers
9k views

Why is OLS estimator of AR(1) coefficient biased?

I am trying to understand why OLS gives a biased estimator of an AR(1) process. Consider $$ \begin{aligned} y_{t} &= \alpha + \beta y_{t-1} + \epsilon_{t}, \\ \epsilon_{t} &\stackrel{iid}{\...
9
votes
2answers
663 views

Is VAR a MANOVA with auto regression?

What are the differences between VAR (vector auto regression) and MANOVA?
8
votes
4answers
746 views

How to create a markov chain with gamma marginal distribution and AR(1) coefficient of $\rho$

I want to generate a synthetic time series. The time series needs to be a markov chain with a gamma marginal distribution and an AR(1) parameter of $\rho$. Can I do this by simply using a gamma ...
8
votes
1answer
218 views

Intuitive explanation/motivation of stationary distribution of a process

Often, in literature, authors have been interested in finding the stationary distribution of a time-series process. For example, consider the following simple AR($1$) process $\{X_t\}$: $$X_t = \alpha ...
7
votes
2answers
2k views

Variance of a smoothed AR(1) process

The query I have relates to calculating the variance of AR(1) processes that are smoothed with a simple moving average. So: In an AR(1) process of the form: $$ X_t=c+\varphi X_{t-1}+\varepsilon_t, $$...
7
votes
1answer
688 views

What is the expected value of the sample variance under a linear regression with omitted variables of an AR(2) process?

Lately, I have been interested in phenomenons related to omission of variables. For example, it can be shown that the expected value of the sample variance under the inclusion of one variable $x_1$ ...
7
votes
2answers
428 views

Writing AR(1) as a MA($\infty$) process

The AR(1) process is $$ X_t = \phi X_{t-1} + \varepsilon_t $$ if we use this formula recursively, we get $$ X_t = \phi(\phi X_{t-2} + \varepsilon_{t-1}) + \varepsilon_t = \phi^2X_{t-2} + \phi\...
7
votes
1answer
2k views

Biased estimates when intercept is included in a linear regression

I am simulating 10000 data-sets, each of length 20, that follow an autoregressive model with lag 1, using the following code: ...
7
votes
1answer
2k views

In spatial regression, what is a spherical autocorrelation structure?

I have a large gridded dataset for the globe (i.e a spherical, wraparound surface) that I'm applying spatial regression to (using a CAR model). I've been using the default autocorrelation function, ...
7
votes
2answers
3k views

Boosted AR for time series forecasting?

I have time series data recorded at multiple locations, stored in a matrix $Y$. I have fit a Vector Autoregressive Model to it which forecasts the data pretty well on a test set. However, if I plot ...
7
votes
2answers
582 views

Lagged dependent variable in linear regression

Recently I read an paper where in a time series data has been modelled according to the equation $$ Y_t=\beta_1 Y_{t−1}+\beta_2X+\varepsilon. $$ OLS was used here (with the ...
7
votes
1answer
323 views

Nonstationary solutions for stationary ARMA equations

By "stationary" I mean "weakly stationary". Consider a "stationary" AR(1) equation: $$X_t=\varphi X_{t-1}+\varepsilon_t,$$ where $t\in\mathbb{Z}$ are discrete time moments, $\varepsilon_t$ a zero-...
6
votes
2answers
7k views

Auto-regression versus linear regression of x(t)-with-t for modelling time series

What difference precisely does autoregression (for AR(p), p=1,2,...) have when compared to linear regression of that time series random variable w.r.t time axis? Explanation with diagrams clarifying ...
6
votes
2answers
8k views

How to write an AR(2) stationary process in the Wold representation

I managed to write an AR(1) process in the Wold representation with help from the geometric series. I am having trouble with a stationary AR(2). How could I do?
6
votes
3answers
10k views

What's a stationary VAR?

What is a stationary VAR (vector autoregression)? Can a VAR with non-stationary variables be stationary? How do you test whether a VAR is stationary or non-stationary? (Example in ...
6
votes
1answer
9k views

AR(1) coefficient is correlation?

Is the ar1 coefficient from an AR(1) model the "first order correlation of the noise" of a time series? I'm using R's aws package and one of the arguments of the <...
6
votes
1answer
1k views

Why is the dickey fuller test different from a simple t-test

I am trying to understand why should there be different distribution for t-statistic, in case of AR model, Dickey-Fuller test For e.g. Say, the model is $Y_t = \beta_lY_{t-1} + \varepsilon_{t}$. ...
6
votes
2answers
2k views

Estimation of unit-root AR(1) model with OLS

Given a random walk $x_t$, $$x_t=x_{t-1}+\varepsilon_t,$$ consider estimating the slope coefficient $\beta$ in $$x_t=\beta x_{t-1}+\varepsilon_t$$ by OLS. This question and the following answer ...
6
votes
1answer
417 views

Independent Bernoulli trials vs markov chain

Original Question Suppose we have a sequence of Bernoulli trials $X_1, X_2, \cdots X_T$ which are ordered in time and may or may not be independent. I am interested in understanding the probability ...
6
votes
2answers
3k views

Poisson with an autoregressive term

I want to fit a fairly "standard" Poisson model, but with an autoregressive term. $N_i \sim \mathrm{Pois}( \lambda_i E_i)$ with $\log \lambda_i = X_i \beta + \delta$ $\delta \sim AR(1)$ $X_i$ is a ...
6
votes
2answers
2k views

What is the reason for not including an intercept term in AR and ARMA models?

In econometric literature it is usually argued that in case of estimating an equation, an intercept term must be always included regardless of its statistical importance because removing the constant ...
6
votes
3answers
3k views

AR(2) model is causal

AR(2) model is: $$X_t=\phi_1X_{t-1}+\phi_2X_{t-2}+W_t$$ Where $W_t\sim N(o,\sigma^2)$ I want to prove AR(2) model is causal. So, I tried as: $$X_t-\phi_1X_{t-1}-\phi_2X_{t-2}=W_t$$ $$\Rightarrow (...
6
votes
1answer
251 views

The distribution of the initial point of an AR process

Consider a stochastic process $\{X_t, t = 1, 2, \ldots\}$ following the model $$X_t = \alpha X_{t-1} + e_t,$$ where $e_t \thicksim f$. Can I say that the distribution of the initial point, $X_1$, ...
6
votes
1answer
6k views

How to interpret coefficients in a vector autoregressive model?

Can I interpret the coefficients in a VAR model in the same way as I do in a normal OLS regression?
6
votes
0answers
152 views

What guarantees the existence of a finite representation of the Wold decomposition? Mechanics and Intuition

Every covariance stationary process can be written as a linear, infinite distributed lag of white noise. In other words, every covariance stationary process has a Wold representation. Then we go on to ...
6
votes
0answers
599 views

Computing Standard Errors in EM algorithm

I'm applying the EM to a hidden markov chain (the $\mathbf{Z}=\{Z_1,...,Z_n\}$ variable), with observations(the $\mathbf{Y}=\{Y_0,...,Y_n\}$ variable) dependent not only on the hidden markov chain, ...
5
votes
4answers
2k views

How to compute the standard error of the mean of an AR(1) process?

I try to compute the standard error of the mean for a demeaned AR(1) process $x_{t+1} = \rho x_t + \varepsilon_{t+1} =\sum\limits_{i=0}^{\infty} \rho^i \varepsilon_{t+1-i}$ Here is what I did: $$ \...
5
votes
1answer
2k views

Why taking the first difference is the same as an AR(1) filter?

Today, my professor said that for highly correlated time series, taking the first differentiation is like applying an AR(1) filter.. Unfortunately I a was not able to ask him after the lecture. I am ...
5
votes
3answers
1k views

How to fit an autoregressive (AR(1)) model with trend and/or seasonality to a time series?

I want to test a model I have on a time series. The model is that the time series adapts to a trend $f(t)$ with a speed $\alpha$. There is also noise in the model. So, the time series is a function ...
5
votes
1answer
1k views

Finding stationary values for an AR(2) process

I have been given the following AR(2) process: $X_t + \phi_1 X_{t-1} + \phi_2 X_{t-2} = \epsilon_t$ and I need to figure out for which values of $\phi_2$ the process is stationary, when I have been ...
5
votes
1answer
109 views

With two restrictions on the parameters, how does an AR(p) process change as we increase p

Suppose we have an AR(1) process with parameter $\theta$ such that $Z_i = \theta Z_{i-1} + \epsilon_i$ I wish to compare this to the general AR(p) process of the form $Z_i = \theta_1 Z_{i-1} + \...
5
votes
1answer
22k views

Step-by-step example of predicting time series with ARIMAX or ARMAX model?

Could someone give me a step-by-step example of time series prediction using ARIMAX or ARMAX model? The example doesn't need to be long or complicated. It could be for example forecasting temperature ...
5
votes
1answer
297 views

Memoryless Property of a Markov Chain of Order 1. Is AR(1) memoryless or of infinite memory?

A stochastic process constitutes a discrete Markov Chain of order 1 if it has the memoryless property, in the sense that the probability that the chain will be in a particular state i, of a finite set ...
5
votes
1answer
854 views

Explanation of the 'free bits' technique for variational autoencoders

I have been reading through a couple of papers on the variational autoencoder model: 'Variational Lossy Autoencoder' and 'Improving Variational Inference With Inverse Autoregressive Flow'. There is ...
5
votes
1answer
299 views

Generate a random variable which follow Gamma distribution and AR(1) process simulatenously

Is it possible to generate numbers from Gamma distribution (with parameters shape=10, scale=15, say) which also follow a AR(1) process, simultaneously? If it's possible, than how to do that?
5
votes
1answer
279 views

A closed form formula for the normalizing constant in standard normal auto-regressive series?

Let $Z_t = c_1Z_{t-1} + c_2Z_{t-2} + ... + c_nZ_{t-n} + c\epsilon_t$ where $Z_t, \epsilon_t \sim \mathtt{N}(0,1)$ are iid variables and $Z_s \sim \mathtt{N}(0,1)$ for all $s$. Given the values of $...
5
votes
1answer
258 views

Is there a convenient form for this large covariance matrix?

Consider the following bivariate vector autoregression: $$X_t=\mu +X_{t-1}A+\varepsilon_t,\ \varepsilon_t \overset{iid}{\sim} MVN(0, V),\ X_t=(X_{1,t},X_{2,t})',$$ where the assumptions on the ...
5
votes
1answer
1k views

Why don't we look at $R^2$ when fitting an autoregressive model?

$R^2$ measures explained variance. In an autoregressive model like AR(k), we are carrying out a linear regression, and as such we would have an $R^2$ and an ...