When we fit a general linear model, we assume that the errors are independent. But why not just assume that the observations are independent? Isn't it equivalent, and (in my view) more intuitive?
$\begingroup$
$\endgroup$
asked Jul 6, 2021 at 18:18
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
The observations are assumed to be dependent!
Assume some regression like $\hat y=2x$.
Then we know to expect $\hat y(0)<\hat y(1)$.
However, even if we know that $\beta=2$, we have no dependence between the errors on observations with $x=0$ and $x=1$.
-
$\begingroup$ You might have to make some qualifications to make this statement true! What if it's a linear model with random group effects? $\endgroup$– dipetkovCommented Jul 19, 2022 at 11:23
$\begingroup$
$\endgroup$
The assumption that the errors are independent means there is no relationship between the residuals of our model (the errors) and the response variable (observations).
You are correct that the observations are also assumed to be independent of each other and the errors should also be independent of each other!
$\begingroup$
$\endgroup$
The observations are not always independent. For instance, there may be auto-correlation across data points that are close together in time/space.
In those cases, we need to specify a model that accounts for the dependence structure of the observations in a statistically adequate way, which will ensure that the residuals from the model will be IID. For example, we could use ARIMA models.
Another option is to continue with standard OLS and possibly non-independent residuals, but to correct the estimates of standard errors, using appropriate estimators, such as Newey-West standard errors
