Least squares estimator in a time series $\{Y_t\}$

Question

Let $\{Y_t\}$ be a stochastic process such that $$\begin{cases}Y_t=\beta x_t+z_t\\z_t=\varepsilon_t+\theta \varepsilon_{t-1}\\\varepsilon_t\sim WN(0,1)\end{cases}$$

where $WN$ means white noise (it's not a probability distribution) with $\mathbb{E}[\varepsilon_t]=0$ and $\text{Var}(\varepsilon_t)=1$. The $x_t$ values are constants not random. Find the least squares estimator of $\beta$ and the variance of estimator.

Question: Is the following correct?

Attempt: What I did is $$Q=\sum (y_t-(\beta x_t+z_t))^2=\sum y_t^2-2y_t(\beta x_t+z_t)+(\beta x_t+z_t)^2$$ $$=\sum y_t^2-2y_t\beta x_t-2y_tz_t+\beta^2x_t^2+2\beta x_tz_t+z_t^2$$ $$\frac{\partial Q}{\partial \beta}=\sum -2y_tx_t+2\beta\sum x_t^2-\sum x_tz_t=0$$ $$\Leftrightarrow \hat{\beta}=\frac{\sum y_tx_t-\sum x_tz_t}{\sum x_t^2}$$

Taking the second derivative $$\frac{\partial^2Q}{\partial\beta\partial\beta}=\sum x_t^2>0\qquad \forall t$$

then $\hat{\beta}$ is a minimum point and is the least square estimator.

$$\text{Var}(\hat{\beta})=\text{Var}\left(\frac{\sum y_tx_t-\sum x_tz_t}{\sum x_t^2}\right)=\frac{1}{(\sum x_t^2)^2}\text{Var}\left(\sum y_tx_t-\sum x_tz_t\right)$$

$$=\frac{1}{(\sum x_t^2)^2}\big(\text{Var}\left(\sum y_tx_t\right)+\text{Var}\left(\sum x_tz_t\right)-2\text{Cov}\left(\sum y_tx_t,\sum x_tz_t)\right)$$

$$=\frac{1}{(\sum x_t^2)^2}\Big(\text{Var}\Big[\sum \beta x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big)\Big]+\text{Var}\Big[\sum x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big)\Big]-2\text{Cov}\Big[\sum \beta x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big),\sum x_t\Big(\varepsilon_t+\theta\varepsilon_{t-1}\Big)\Big]\Big)$$

Is there any mistake in the estimator?

Answer 1

Your derivation of the ordinary least-squares (OLS) estimator is incorrect, since you have treated the error term $z_t$ as if it were part of the regression. Fixing this should give you the standard formula for the OLS estimator for a regression with a single variable and known zero mean:

$$\hat{\beta} = \frac{\sum x_t Y_t}{\sum x_t^2}.$$

To obtain the variance of this estimator, we first note that your error series $\boldsymbol{Z} \equiv \{ Z_t | t \in \mathbb{Z} \}$ is an MA($1$) process with covariance terms:

$$\mathbb{C}(Z_t, Z_{t+k}) = \begin{cases} 1+\theta^2 & \text{for } k = 0, \\[4pt] \theta & \text{for } k = 1, \\[4pt] 0 & \text{for } k > 1. \end{cases}$$

Since $Y_t = Z_t + \text{const}$ this gives you:

$$\begin{equation} \begin{aligned} \mathbb{V} \Bigg( \sum_{t=1}^n x_t Y_t \Bigg) = \mathbb{V} \Bigg( \sum_{t=1}^n x_t Z_t \Bigg) &= \sum_{t=1}^n \sum_{r=1}^n x_t x_r \mathbb{C} (Z_t, Z_r) \\[6pt] &= (1+\theta^2) \sum_{t=1}^n x_t^2 + 2 \theta \sum_{t=1}^{n-1} x_t x_{t+1}. \end{aligned} \end{equation}$$

You then have:

$$\mathbb{V}(\hat{\beta}) = \frac{(1+\theta^2) \sum_{t=1}^n x_t^2 + 2 \theta \sum_{t=1}^{n-1} x_t x_{t+1}}{(\sum_{t=1}^n x_t^2)^2}.$$