How to manually fit MA1 model with OLS? We can manually fit AR1 model using linear model, as discussed here.
But how to manually fit MA1 model? following code seems incorrect .., but how can we write the explanatory variable $w_t$ and $w_{t-1}$ ?
set.seed(0)
ts.sim <- arima.sim(list(order = c(0,0,1), ma = c(0.3)), n = 1e4)
y=ts.sim
lm(y~1)

 A: Consider an MA(q) model
$$
y_t=c+\varepsilon_t+\theta_1\varepsilon_{t-1}+\dots+\theta_q\varepsilon_{t-q}.
$$
You cannot fit the model with OLS (which the lm function is doing). Since the regressors $\varepsilon_{t-i}$ for $i=1,\dots,q$ are unobserved, you cannot construct the $X$ matrix* neeeded for the OLS solution $\beta=(c,\theta_1,\dots,\theta_q)=(X^\top X)^{-1}X^\top y$. The model can be estimated using maximum likelihood via Kalman filtering.
*In this model, for the OLS estimator to work the regressor matrix would need to be $X=(\mathbf{1},\varepsilon_{t-1},\dots,\varepsilon_{t-q})$.
A: You can estimate an MA model using OLS, but you need to do it iteratively.
Consider an MA(1) without intercept:
$$
y_t = \epsilon_t + \theta\epsilon_{t-1}.
$$
The algorithm is:
Set an initial value, $\theta^{(0)}=0$. Fix $\hat{\epsilon}_0=0$. For $j=1, \dots, n_{iter}$ do:

*

*Compute the current estimate of the error terms,
$$
\hat{\epsilon}_t^{(j-1)}=y_t-\hat{\theta}^{(j-1)}\hat{\epsilon}_{t-1}^{(j-1)}
$$

*Run a regression of $y_t$ on $\hat{\epsilon}_{t-1}^{(j-1)}$ and update $\hat{\theta}^{(j)}$

*Repeat

Here is some R code and a figure showing how the estimate evolves in 10 iterations:
set.seed(1)   

N <- 1000
nIter <- 10

eps <- rnorm(N+1)
theta <- 0.5
y <- eps[2:(N+1)] + theta * eps[1:N]

epsEst <- numeric(N+1)
epsEst[1] <- 0
thetaEst <- 0
thetaEstStorage <- numeric(nIter)
thetaEstStorage[1] <- thetaEst
for (j in 2:nIter) {
  for (i in 1:N) {
    epsEst[i+1] <- y[i] - thetaEst * epsEst[i]
  }
  mod <- lm(y ~ 0 + epsEst[1:N])
  thetaEst <- coef(mod)
  thetaEstStorage[j] <- thetaEst
}

ts.plot(thetaEstStorage, ylim = c(-1, 1))
abline(h = theta, col = "red")


