1
$\begingroup$

We can manually fit AR1 model using linear model, as discussed here.

But how to manually fit MA1 model? following code seems incorrect .., but how can we write the explanatory variable $w_t$ and $w_{t-1}$ ?

set.seed(0)
ts.sim <- arima.sim(list(order = c(0,0,1), ma = c(0.3)), n = 1e4)
y=ts.sim
lm(y~1)
$\endgroup$

2 Answers 2

2
$\begingroup$

You can estimate an MA model using OLS, but you need to do it iteratively.

Consider an MA(1) without intercept:

$$ y_t = \epsilon_t + \theta\epsilon_{t-1}. $$

The algorithm is:

Set an initial value, $\theta^{(0)}=0$. Fix $\hat{\epsilon}_0=0$. For $j=1, \dots, n_{iter}$ do:

  1. Compute the current estimate of the error terms, $$ \hat{\epsilon}_t^{(j-1)}=y_t-\hat{\theta}^{(j-1)}\hat{\epsilon}_{t-1}^{(j-1)} $$
  2. Run a regression of $y_t$ on $\hat{\epsilon}_{t-1}^{(j-1)}$ and update $\hat{\theta}^{(j)}$
  3. Repeat

Here is some R code and a figure showing how the estimate evolves in 10 iterations:

set.seed(1)   

N <- 1000
nIter <- 10

eps <- rnorm(N+1)
theta <- 0.5
y <- eps[2:(N+1)] + theta * eps[1:N]

epsEst <- numeric(N+1)
epsEst[1] <- 0
thetaEst <- 0
thetaEstStorage <- numeric(nIter)
thetaEstStorage[1] <- thetaEst
for (j in 2:nIter) {
  for (i in 1:N) {
    epsEst[i+1] <- y[i] - thetaEst * epsEst[i]
  }
  mod <- lm(y ~ 0 + epsEst[1:N])
  thetaEst <- coef(mod)
  thetaEstStorage[j] <- thetaEst
}

ts.plot(thetaEstStorage, ylim = c(-1, 1))
abline(h = theta, col = "red")

enter image description

$\endgroup$
4
  • 1
    $\begingroup$ +1. I would not call this OLS, though; IRWLS would be more appropriate. I think the conceptual difference is substantial, especially since we are dealing with unobservable series as regressors here. $\endgroup$ Jan 21, 2021 at 16:50
  • $\begingroup$ @RichardHardy Sure, I see your point and it's definitely not standard OLS estimation. My goal was just to illustrate that if the OP was asking how to estimate an MA model using only OLS, then that is certaintly possible. $\endgroup$
    – hejseb
    Jan 22, 2021 at 7:19
  • $\begingroup$ Yes, and you did that well. $\endgroup$ Jan 22, 2021 at 7:41
  • $\begingroup$ What does IRWLS stand for? $\endgroup$
    – asmaier
    Jul 20, 2022 at 15:30
2
$\begingroup$

Consider an MA(q) model $$ y_t=c+\varepsilon_t+\theta_1\varepsilon_{t-1}+\dots+\theta_q\varepsilon_{t-q}. $$ You cannot fit the model with OLS (which the lm function is doing). Since the regressors $\varepsilon_{t-i}$ for $i=1,\dots,q$ are unobserved, you cannot construct the $X$ matrix* neeeded for the OLS solution $\beta=(c,\theta_1,\dots,\theta_q)=(X^\top X)^{-1}X^\top y$. The model can be estimated using maximum likelihood via Kalman filtering.

*In this model, for the OLS estimator to work the regressor matrix would need to be $X=(\mathbf{1},\varepsilon_{t-1},\dots,\varepsilon_{t-q})$.

$\endgroup$
1
  • 1
    $\begingroup$ This has been discussed in multiple posts before, so if you find a duplicate, you may mark the question accordingly. $\endgroup$ Jan 21, 2021 at 13:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.