3
$\begingroup$

In this post, one of the answers provides the following information about the assumptions of linear regression in the case of random design (as opposed to fixed design):

The usual regression model is $Y=X\beta+\varepsilon$ and the assumpitons are:

  • $E[\varepsilon|X]=0$
  • Homoscedasticity, $E[\varepsilon^2|X]=\sigma^2$
  • No serial correlation, $E[\varepsilon_i,\varepsilon_j|X]=0$

Note that there is a subscript on the error terms in the last assumption. Does this mean this assumption is not relevant to the population variables, but only to the sample data?

To make it clearer what I'm asking, consider the linear regression model with respect to population variables vs sample variables. Denote by $X^p$, $Y^p$, and $\varepsilon^p$ the population variables for which the regression model holds: $$ Y^p = f(X^p) + \varepsilon^p. $$

Consider drawing $n$ samples from the population $(X_1^s,Y_1^s),(X_2^s,Y_2^s),\dots,(X_n^s,Y_n^s)$. The linear regression model applied to the samples is $$ Y_i^s = f(X_i^s) + \varepsilon_i^s, \quad \quad i=1,2,\dots,n. $$ where the errors are $\varepsilon_i$ all have the same CDF as the population error $\varepsilon^p$. In the samples we have multiple error observations $\varepsilon_i$ so the 'no serial correlation' assumption makes sense. But for the population model, we just have a single random variable for the error $\varepsilon^p$, so it seems we can't speak of 'no serial correlation' with respect to the population?

I have assumed that we can frame the linear regression model with respect to both the sample data and the population we have drawn the sample from. However, if I have made some fundamental mistake in drawing a distinction between the regression model applied to the population vs the sample please let me know, I am still getting to grips with the details of statistical models.

$\endgroup$
  • 4
    $\begingroup$ The sense of "serial correlation" is ambiguous here: does it mean serially in the order the data were collected, as it often does in such circumstances, or does it mean serially in an order established by one of the regressor variables, as it does in time series analysis (for instance)? The latter would have a clear meaning in the population but the former (obviously) would not. $\endgroup$ – whuber Sep 25 at 14:32
  • 1
    $\begingroup$ "The usual regression model is Y=Xβ+ε and the assumpitons are..."---the "usual assumption" for large sample inference for the linear regression model does not require no serial correlation (not at all, really), or conditional homoskedasticity for that matter. "Does this mean this assumption is not relevant to the population variables, but only to the sample data?"---the stated condition, $E[\varepsilon_i,\varepsilon_j|X]=0$, is a population assumption. $\endgroup$ – Michael Sep 26 at 1:31
  • 2
    $\begingroup$ @ManUtdBloke, if you can index the elements of the sample by $i$ and $j$, why would you think you cannot index the elements of the population accordingly? $\endgroup$ – Richard Hardy Sep 28 at 10:05
  • 1
    $\begingroup$ The reference to "serial correlation" refers to a post in which this term is misused. Because the implicit quantifiers in that post are all distinct pairs $i,j,$ this assumption has nothing to do with correlation: it simply means that no two errors are correlated. $\endgroup$ – whuber Sep 28 at 13:07
  • 2
    $\begingroup$ In that case his language is incorrect! Serial correlation is clearly and authoritatively defined in most time series textbooks, where it concerns the structure of correlations among a set of random variables that are ordered along a line. That's literally what "serial" means. Indeed, in the multiple regression setting (which Salkind appears to be considering on that page) there is no natural ordering of the data, making the term "serial" meaningless and therefore superfluous. $\endgroup$ – whuber Sep 29 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.