Let $\{Y_t\}$ be a stochastic process such that $$\begin{cases}Y_t=\beta x_t+z_t\\z_t=\varepsilon_t+\theta \varepsilon_{t-1}\\\varepsilon_t\sim WN(0,1)\end{cases}$$

where $WN$ means white noise (it's not a probability distribution) with $\mathbb{E}[\varepsilon_t]=0$ and $\text{Var}(\varepsilon_t)=1$. The $x_t$ values are constants not random. Find the least squares estimator of $\beta$ and the variance of estimator.

Question: Is the following correct?

Attempt: What I did is $$Q=\sum (y_t-(\beta x_t+z_t))^2=\sum y_t^2-2y_t(\beta x_t+z_t)+(\beta x_t+z_t)^2$$ $$=\sum y_t^2-2y_t\beta x_t-2y_tz_t+\beta^2x_t^2+2\beta x_tz_t+z_t^2$$ $$\frac{\partial Q}{\partial \beta}=\sum -2y_tx_t+2\beta\sum x_t^2-\sum x_tz_t=0$$ $$\Leftrightarrow \hat{\beta}=\frac{\sum y_tx_t-\sum x_tz_t}{\sum x_t^2}$$

Taking the second derivative $$\frac{\partial^2Q}{\partial\beta\partial\beta}=\sum x_t^2>0\qquad \forall t$$

then $\hat{\beta}$ is a minimum point and is the least square estimator.

$$\text{Var}(\hat{\beta})=\text{Var}\left(\frac{\sum y_tx_t-\sum x_tz_t}{\sum x_t^2}\right)=\frac{1}{(\sum x_t^2)^2}\text{Var}\left(\sum y_tx_t-\sum x_tz_t\right)$$

$$=\frac{1}{(\sum x_t^2)^2}\big(\text{Var}\left(\sum y_tx_t\right)+\text{Var}\left(\sum x_tz_t\right)-2\text{Cov}\left(\sum y_tx_t,\sum x_tz_t)\right)$$

$$=\frac{1}{(\sum x_t^2)^2}\Big(\text{Var}\Big[\sum \beta x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big)\Big]+\text{Var}\Big[\sum x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big)\Big]-2\text{Cov}\Big[\sum \beta x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big),\sum x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big)\Big]\Big)$$

Is there any mistake in the estimator?

  • 2
    $\begingroup$ I think $z_t$ is unobservable, so your estimator that includes $z_t$ is infeasible. $\endgroup$ Sep 16, 2016 at 5:40
  • $\begingroup$ Is WN following a Normal Distribution? If so your process is a Moving-Average process $\endgroup$
    – Ferdi
    Sep 16, 2016 at 6:59
  • $\begingroup$ Roland, the mistake is that your estimator is infeasible. You physically cannot apply it. What else do you want to learn? $\endgroup$ Sep 23, 2016 at 17:26
  • $\begingroup$ What properties do you want your estimator to have? Some models just cannot have good OLS estimators. OLS is not a universal estimation method. $\endgroup$ Sep 23, 2016 at 17:47
  • 1
    $\begingroup$ @RichardHardy But the estimator $\hat{\beta}=(x'x)^{-1}x'y$ is the least square estimator when you have random errors, but in this case there is no error but a noise. It will be the same? $\endgroup$
    – user72621
    Sep 24, 2016 at 13:19

1 Answer 1


Your derivation of the ordinary least-squares (OLS) estimator is incorrect, since you have treated the error term $z_t$ as if it were part of the regression. Fixing this should give you the standard formula for the OLS estimator for a regression with a single variable and known zero mean:

$$\hat{\beta} = \frac{\sum x_t Y_t}{\sum x_t^2}.$$

To obtain the variance of this estimator, we first note that your error series $\boldsymbol{Z} \equiv \{ Z_t | t \in \mathbb{Z} \}$ is an MA($1$) process with covariance terms:

$$\mathbb{C}(Z_t, Z_{t+k}) = \begin{cases} 1+\theta^2 & \text{for } k = 0, \\[4pt] \theta & \text{for } k = 1, \\[4pt] 0 & \text{for } k > 1. \end{cases}$$

Since $Y_t = Z_t + \text{const}$ this gives you:

$$\begin{equation} \begin{aligned} \mathbb{V} \Bigg( \sum_{t=1}^n x_t Y_t \Bigg) = \mathbb{V} \Bigg( \sum_{t=1}^n x_t Z_t \Bigg) &= \sum_{t=1}^n \sum_{r=1}^n x_t x_r \mathbb{C} (Z_t, Z_r) \\[6pt] &= (1+\theta^2) \sum_{t=1}^n x_t^2 + 2 \theta \sum_{t=1}^{n-1} x_t x_{t+1}. \end{aligned} \end{equation}$$

You then have:

$$\mathbb{V}(\hat{\beta}) = \frac{(1+\theta^2) \sum_{t=1}^n x_t^2 + 2 \theta \sum_{t=1}^{n-1} x_t x_{t+1}}{(\sum_{t=1}^n x_t^2)^2}.$$

  • $\begingroup$ Dear @Ben I have a question on unit root ADF test with quadratic trend. Can you please look at my question. I will be very happy and all helps will be much appreciated. stats.stackexchange.com/questions/589970/… $\endgroup$
    – 1190
    Oct 2, 2022 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.