1
$\begingroup$

For various reasons we often assume that there is independence across observations in linear regression models. When does that assumption hold? Only when the data is collected by random sampling?

I could only find descriptions of it here on stackexchange. But I am interested in the condition. Is random sampling a necessary condition to talk about i.i.d. random variables?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ No. Although random sampling can be helpful for justifying the iid assumption, it is just that: a modeling assumption. You can apply it any time you like, but YMMV depending on how realistic that assumption is. $\endgroup$
    – whuber ♦
    Commented Dec 29, 2023 at 14:18

2 Answers 2

Reset to default
3
$\begingroup$

Short answer: No.

Longer answer: Random sampling is a different thing from independence. Take the archeypical sample of "sophomore college students in a survey course". This is not a random sample of any population (not even all sophomores in the country). But there's no reason to thing that the errors in any model would not be independent, at least if you just take that class as your sample.

For example, if you are regressing weight on height, then there is no reason to think that the error for Anjali should be related to the error for Jill.

Improve this answer
$\endgroup$
3
$\begingroup$

Consider this counterexample showing that independence can hold without random sampling:

Let $N_1, N_2 \sim \mathcal{N}(0, 1)$ independently, and let $X_1 := N_1$ and $X_2 := \beta X_2 + N_2$.

Given observations $(X_1^{(i)}, X_2^{(i)})_{i=1}^n$, one could choose only such observations where $-1 < X_1 < 1$, making this a non-random sample. Still, the observations would be just as independent as before.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.