I am trying to understand why OLS gives a biased estimator of an AR(1) process. Consider $$ \begin{aligned} y_{t} &= \alpha + \beta y_{t-1} + \epsilon_{t}, \\ \epsilon_{t} &\stackrel{iid}{\sim} N(0,1). \end{aligned} $$ In this model, strict exogeneity is violated, i.e. $y_t$ and $\epsilon_t$ are correlated but $y_{t-1}$ and $\epsilon_t$ are uncorrelated. But if this is true, then why does the following simple derivation not hold? $$ \begin{aligned} \text{plim} \ \hat{\beta} &= \frac{\text{Cov}(y_{t},y_{t-1})}{\text{Var}(y_{t-1})} \\ &=\frac{\text{Cov}(\alpha + \beta y_{t-1}+\epsilon_{t}, y_{t-1})}{\text{Var}(y_{t-1})} \\ &= \beta+ \frac{\text{Cov}(\epsilon_{t}, y_{t-1})}{\text{Var}(y_{t-1})} \\ &=\beta. \end{aligned} $$

  • $\begingroup$ There have been a few related questions at Cross Validated. You could benefit from looking them up. $\endgroup$ – Richard Hardy Oct 15 '16 at 14:12
  • $\begingroup$ I saw them, but they did not really help me. I found a proof and simulations that show this result. What I am interested in is what is wrong with my reasoning above. $\endgroup$ – Florestan Oct 15 '16 at 17:45
  • 1
    $\begingroup$ When you are using $\text{plim}$, aren't you addressing consistency rather than (un)biasedness? For (un)biasedness you should be using expectations. $\endgroup$ – Richard Hardy Oct 15 '16 at 18:04
  • $\begingroup$ You are completely right, that could solve the puzzle. So if the equation above does not hold without a plim, then it would not contradict the biasedness of OLS in small samples and show the consistency of OLS at the same time. Though I am a bit unsure: Does this covariance over variance formula really only hold for the plim and not also in expectation? Thanks a lot already! $\endgroup$ – Florestan Oct 15 '16 at 19:00
  • 1
    $\begingroup$ OLS estimator itself does not involve any $\text{plim}$s, you should just look at expectations in finite samples. $\endgroup$ – Richard Hardy Oct 15 '16 at 19:44

As essentially discussed in the comments, unbiasedness is a finite sample property, and if it held it would be expressed as

$$E (\hat \beta ) = \beta$$

(where the expected value is the first moment of the finite-sample distribution)

while consistency is an asymptotic property expressed as

$$\text{plim} \hat \beta = \beta$$

The OP shows that even though OLS in this context is biased, it is still consistent.

$$E (\hat \beta ) \neq \beta\;\;\; \text{but}\;\;\; \text{plim} \hat \beta = \beta$$

No contradiction here.

| cite | improve this answer | |

@Alecos nicely explains why a correct plim and unbiasedbess are not the same. As for the underlying reason why the estimator is not unbiased, recall that unbiasedness of an estimator requires that all error terms are mean independent of all regressor values, $E(\epsilon|X)=0$.

In the present case, the regressor matrix consists of the values $y_1,\ldots,y_{T-1}$, so that - see mpiktas' comment - the condition translates into $E(\epsilon_s|y_1,\ldots,y_{T-1})=0$ for all $s=2,\ldots,T$.

Here, we have

\begin{equation*} y_{t}=\beta y_{t-1}+\epsilon _{t}, \end{equation*} Even under the assumption $E(\epsilon_{t}y_{t-1})=0$ we have that \begin{equation*} E(\epsilon_ty_{t})=E(\epsilon_t(\beta y_{t-1}+\epsilon _{t}))=E(\epsilon _{t}^{2})\neq 0. \end{equation*} But, $y_t$ is also a regressor for future values in ain AR model, as $y_{t+1}=\beta y_{t}+\epsilon_{t+1}$.

| cite | improve this answer | |
  • 3
    $\begingroup$ I would add the clarification that $E(\varepsilon | X)$ in this case translates to $E(\varepsilon_s|y_{1},...,y_T)$ for each $s$. Then the further discussion becomes a bit clearer. $\endgroup$ – mpiktas Oct 19 '16 at 6:57
  • $\begingroup$ good point, I made an edit $\endgroup$ – Christoph Hanck Oct 19 '16 at 8:02

Expanding on two good answers. Write down the OLS estimator:

$$\hat\beta =\beta + \frac{\sum_{t=2}^Ty_{t-1}\varepsilon_t}{\sum_{t=2}^Ty_{t-1}^2}$$

For unbiasedness we need


But for that we need that $E(\varepsilon_t|y_{1},...,y_{T-1})=0,$ for each $t$. For AR(1) model this clearly fails, since $\varepsilon_t$ is related to the future values $y_{t},y_{t+1},...,y_{T}$.

| cite | improve this answer | |
  • $\begingroup$ Just to check whether I got it right: The problem is not the numerator, for each t $y_{t-1}$ and $\epsilon_{t}$ are uncorrelated. The problem is the denominator that features higher t's such that there is correlation between numerator and denominator so that I cannot take the expectation within the sum of the numerator (under strict exogeneity I could do so?!). Is that the correct mathematical intuition? $\endgroup$ – Florestan Oct 20 '16 at 21:33
  • $\begingroup$ Yes that is correct intuition. Note that strict exogeneity is not possible in this case, but for unbiasedness strict exogeneity becomes a requirement. $\endgroup$ – mpiktas Oct 21 '16 at 7:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.