I am trying to understand why OLS gives a biased estimator of an AR(1) process. Consider $$ \begin{aligned} y_{t} &= \alpha + \beta y_{t-1} + \epsilon_{t}, \\ \epsilon_{t} &\stackrel{iid}{\sim} N(0,1). \end{aligned} $$ In this model, strict exogeneity is violated, i.e. $y_t$ and $\epsilon_t$ are correlated but $y_{t-1}$ and $\epsilon_t$ are uncorrelated. But if this is true, then why does the following simple derivation not hold? $$ \begin{aligned} \text{plim} \ \hat{\beta} &= \frac{\text{Cov}(y_{t},y_{t-1})}{\text{Var}(y_{t-1})} \\ &=\frac{\text{Cov}(\alpha + \beta y_{t-1}+\epsilon_{t}, y_{t-1})}{\text{Var}(y_{t-1})} \\ &= \beta+ \frac{\text{Cov}(\epsilon_{t}, y_{t-1})}{\text{Var}(y_{t-1})} \\ &=\beta. \end{aligned} $$
$\begingroup$
$\endgroup$
5
-
$\begingroup$ There have been a few related questions at Cross Validated. You could benefit from looking them up. $\endgroup$– Richard HardyCommented Oct 15, 2016 at 14:12
-
$\begingroup$ I saw them, but they did not really help me. I found a proof and simulations that show this result. What I am interested in is what is wrong with my reasoning above. $\endgroup$– FlorestanCommented Oct 15, 2016 at 17:45
-
2$\begingroup$ When you are using $\text{plim}$, aren't you addressing consistency rather than (un)biasedness? For (un)biasedness you should be using expectations. $\endgroup$– Richard HardyCommented Oct 15, 2016 at 18:04
-
$\begingroup$ You are completely right, that could solve the puzzle. So if the equation above does not hold without a plim, then it would not contradict the biasedness of OLS in small samples and show the consistency of OLS at the same time. Though I am a bit unsure: Does this covariance over variance formula really only hold for the plim and not also in expectation? Thanks a lot already! $\endgroup$– FlorestanCommented Oct 15, 2016 at 19:00
-
1$\begingroup$ OLS estimator itself does not involve any $\text{plim}$s, you should just look at expectations in finite samples. $\endgroup$– Richard HardyCommented Oct 15, 2016 at 19:44
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
As essentially discussed in the comments, unbiasedness is a finite sample property, and if it held it would be expressed as
$$E (\hat \beta ) = \beta$$
(where the expected value is the first moment of the finite-sample distribution)
while consistency is an asymptotic property expressed as
$$\text{plim} \hat \beta = \beta$$
The OP shows that even though OLS in this context is biased, it is still consistent.
$$E (\hat \beta ) \neq \beta\;\;\; \text{but}\;\;\; \text{plim} \hat \beta = \beta$$
No contradiction here.
answered Oct 15, 2016 at 21:07
$\begingroup$
$\endgroup$
4
@Alecos nicely explains why a correct plim and unbiasedbess are not the same. As for the underlying reason why the estimator is not unbiased, recall that unbiasedness of an estimator requires that all error terms are mean independent of all regressor values, $E(\epsilon|X)=0$.
In the present case, the regressor matrix consists of the values $y_1,\ldots,y_{T-1}$, so that - see mpiktas' comment - the condition translates into $E(\epsilon_s|y_1,\ldots,y_{T-1})=0$ for all $s=2,\ldots,T$.
Here, we have
\begin{equation*} y_{t}=\beta y_{t-1}+\epsilon _{t}, \end{equation*} Even under the assumption $E(\epsilon_{t}y_{t-1})=0$ we have that \begin{equation*} E(\epsilon_ty_{t})=E(\epsilon_t(\beta y_{t-1}+\epsilon _{t}))=E(\epsilon _{t}^{2})\neq 0. \end{equation*} But, $y_t$ is also a regressor for future values in ain AR model, as $y_{t+1}=\beta y_{t}+\epsilon_{t+1}$.
answered Oct 19, 2016 at 6:31
-
3$\begingroup$ I would add the clarification that $E(\varepsilon | X)$ in this case translates to $E(\varepsilon_s|y_{1},...,y_T)$ for each $s$. Then the further discussion becomes a bit clearer. $\endgroup$– mpiktasCommented Oct 19, 2016 at 6:57
-
$\begingroup$ good point, I made an edit $\endgroup$ Commented Oct 19, 2016 at 8:02
-
$\begingroup$ The last equation is wrong, $E(\epsilon_t y_t) = E(\epsilon_t (\beta y_{t-1} + \epsilon_{t-1})) = 0$ $\endgroup$ Commented Sep 26, 2023 at 20:31
-
$\begingroup$ No, see the definition of the AR(1) process in the display above which I plugged in there. $y_t$ is the combination of its lagged value $y_{t-1}$ plus the current innovaton $\epsilon_t$. $\endgroup$ Commented Sep 27, 2023 at 4:37
$\begingroup$
$\endgroup$
5
Expanding on two good answers. Write down the OLS estimator:
$$\hat\beta =\beta + \frac{\sum_{t=2}^Ty_{t-1}\varepsilon_t}{\sum_{t=2}^Ty_{t-1}^2}$$
For unbiasedness we need
$$E\left[\frac{\sum_{t=2}^Ty_{t-1}\varepsilon_t}{\sum_{t=2}^Ty_{t-1}^2}\right]=0.$$
But for that we need that $E(\varepsilon_t|y_{1},...,y_{T-1})=0,$ for each $t$. For AR(1) model this clearly fails, since $\varepsilon_t$ is related to the future values $y_{t},y_{t+1},...,y_{T}$.
-
2$\begingroup$ Just to check whether I got it right: The problem is not the numerator, for each t $y_{t-1}$ and $\epsilon_{t}$ are uncorrelated. The problem is the denominator that features higher t's such that there is correlation between numerator and denominator so that I cannot take the expectation within the sum of the numerator (under strict exogeneity I could do so?!). Is that the correct mathematical intuition? $\endgroup$ Commented Oct 20, 2016 at 21:33
-
1$\begingroup$ Yes that is correct intuition. Note that strict exogeneity is not possible in this case, but for unbiasedness strict exogeneity becomes a requirement. $\endgroup$– mpiktasCommented Oct 21, 2016 at 7:01
-
$\begingroup$ Do you know what the finite sample distribution is of OLS estimates for AR(1) (assuming Gaussian driving noise)? Thank you $\endgroup$ Commented Nov 13, 2020 at 10:43
-
$\begingroup$ Your equation for the OLS estimator is wrong, you didn't center the $y$s correctly. $\endgroup$ Commented Sep 26, 2023 at 20:33
-
$\begingroup$ @Florestan the numerator is problematic too, because $\epsilon_t$ is strongly correlated with the means of the $y$s, $\sum^T_{t=2} y_{t-1} / (T -1)$. $\endgroup$ Commented Sep 26, 2023 at 20:35