Regression assumption violation: lagged dependent variable as regressor

Question

I am studying regression and a bit lost with conceptually understanding why having an independent variable correlated with the error term is a regression assumption violation.

Just to give more context... I am looking into model specifications errors related to Time Series, where a lagged value of the dependent variable is included as an independent variable. And it's said that if we include such lagged dependent variable as independent, then it will be correlated with the error term, thus violating a regression assumption.

Answer 1

Hi: If you write the lagged independent variable in a different manner, you will see why the OLS's optimality breaks down. I will use the AR(1) only because I'm used to that notation and it's a regression with lagged dependent variable.

(1) $y_t = \phi y_{t-1} + \epsilon_t $

Similarly,

(2) $y_{t-1} = \phi y_{t-2} + \epsilon_{t-1} $

Similarly again,

(2) $y_{t-2} = \phi y_{t-3} + \epsilon_{t-2} $

So, since $y_{t-1}$ has $\epsilon_{t-1}$ in it and since $y_{t-2}$ has $\epsilon_{t-2}$ in it, this means that $y_{t}$ is going to have $\phi \epsilon_{t-1}$ in it and $\phi^2 \epsilon_{t-2}$ in it and so on and so forth.

So, what the regression really turns out to be is a regression of y_{t} on sums of error terms with decreasing weights of $\phi$ raised to a power :

$y_{t} = \sum_{i=1}^{t} \phi^{t-i} \epsilon_{i} + \epsilon_t$

So, to tie it in with the answer of @Au p, in the lagged dependent variable case, the regressor is not acting as random error but rather as the weighted regressor itself which is a serious violation of the classical assumptions.

#===================================================================

EDIT: I just saw that Christopher Hanck link has some very nice answers. This one has yet another way of looking at it so could possibly still be helpful. At the same time, if I had read that link first, I probably would not have constructed this answer.

Answer 2

In any statistical analysis, such as regressions(inference), there must be randomness entropy that is considered to be not measurable. If not, then what is the point to study already known patterns. The idea is we can discover one.

That being said, we can move towards the Gauss-Markov theorem, which states that if your linear regression model satisfies the first six classical assumptions, then ordinary least squares (OLS) regression produces unbiased estimates that have the smallest variance of all possible linear estimators.

Therefore, the regression model will try to avoid bias on all it's estimators.

Moreover, if you take a look at Jim Frost approach https://statisticsbyjim.com/regression/ols-linear-regression-assumptions/ You will see that a correlation (not causation) between an independent term and the error term, breaks the notion of "unpredictable random error" which can also be called as "random error"(alone because of it randomness...) or "entropic random error".

However, in practice, I can assure you, this happens ... If you run a predictive model on real data or a monte carlo simulation over fake data, you will find correlations between estimates and error all the time.