Correct Notation for Exogeneity Assumption in the Classical Linear Regression Model

Question

Suppose we have a bivariate population regression model:

$$y_i = \beta_1 + \beta_2x_i + \varepsilon_i$$

Among the assumptions of the CLRM are:

$\quad$ Homoscedasticity: $Var(\varepsilon_i) = \sigma^2\quad\forall i$

$\quad$ No Autocorrelation: $Cov(\varepsilon_i,\varepsilon_j) = 0\quad\forall i,j$

In both of the above, the subscripts indexing units within the population are clearly needed since these assumptions concern relations between different units.

Question: For the exogeneity assumption of CLRM (and using similar notation in terms of individual variables, not vectors or matrices) which of the following (or perhaps something else) is the correct statement?

$$Cov(x,\varepsilon) = 0\qquad\qquad (1)$$

$$Cov(x_i,\varepsilon_i) = 0\quad \forall i \qquad(2)$$

Formulation (1) seems to imply a single calculation of covariance based on variation in $x$ and in $\varepsilon$ over the different units. It seems to suggest that for each unit $i$ there is just one value of $x_i$ and one value of $\varepsilon_i$. But the latter seems wrong: $\varepsilon_i$ is a random variable for each $i$ (as illustrated by the homoscedasticity and no autocorrelation assumptions above which otherwise would not make sense).

Formulation (2) seems to imply a separate calculation of covariance for each $i$, and to treat both $x_i$ and $\varepsilon_i$ as random variables. That could make sense in terms of repeated measurements of the same units that change over time, but that seems to assume a panel data model.

I wonder if my difficulty here lies in the interpretation of the indexing of units. Perhaps $i$ should be thought of not as indexing specific units but as meaning “the $i$’th unit randomly selected”, where the selection is from the full population?

Answer 1

First , let me clear this out, $i$ is definitely an index variable to denote specific units (that is, observations), it is not "the $i$th unit randomly selected". Also it does not have to do anything with repeated mesurements. It is a bivariate measurement taken all at a time.Assume we want to predict weight of a child, based on his height. So we collect height and weight data of say 30 children, that is in mathematical expression,our data has the form, $$ (X_i, Y_i)\; \forall \; i, \quad i = 1(1)30, $$ We can calculate the regression formula from this data and apply it to predict that child's weight (this is a trivial example of regression).

Also if we apply it to all the current sample points, then the $ observed \; Y_i - predicted \; Y_i $ will give $e_i$ for each $i$.These $e_i$s will serve as the proxies of the $\epsilon_i$s as the latter is unobservable. So yes $\epsilon_i$ s are random and can take any values but for a given sample $e_i$ s are fixed.So we can calculate the covariance of the $X_i$ and the $e_i$ values in real data.

Now coming to your point of CLRM assumptions, the given notation is in terms of individual variables and not matrices or vectors,that is they both are single variable, hence, the first formula $$Cov(x, \epsilon) = 0 ,\; $$ is correct. It means that the explanatory variable is uncorrelated with the error variable.Actually, $\epsilon$ is an unobservable random variable and hence we cannot 'calculate' the 'true covariance'.However we can check whether the observed $e_i$s and $x_i$ s covariance is zero, in real data (or at least very close to zero). Note that this assumption is vital because if the explanatory variable and the error variable is indeed correlated then we cannot separate out their individual effects on our study variable Y, and our model will not be useful.

Answer 2

Answering my own question following further study.

Note first that the regressor may be either fixed or stochastic. The regressor would be fixed if, in an experimental context, the researcher determines values of $x$ and then observes the corresponding values of $y$. If the regressor is fixed, it is not a random variable and therefore there can be no covariance with the disturbance term, covariance being (see here) a relation between random variables. So the question only arises when the regressor is stochastic.

Assuming the regressor is stochastic, how are we to interpret the subscripts in the population regression model? One interpretation is that it refers to a randomly selected unit within the population of interest, with $x_i$, $y_i$ and $\varepsilon_i$ being corresponding values for the same unit. This interpretation seems to be implicit in Wooldridge (A) which describes such a model with subscripts as “the population model for a generic draw from the population”. On this interpretation, $\varepsilon_i$ is a random variable only in the sense that its value depends on which unit is drawn. In this sense, the probability distribution of $\varepsilon_i$ is necessarily the same for each draw. Thus the formula for homoscedasticity is trivially satisfied, while that for no autocorrelation is trivially unsatisfied (the covariance of any variable with itself being its variance which, given random variation in the population, will not be zero).

Another interpretation is that the subscript refers to a sub-population defined by a particular value of the regressor. This interpretation may be found in Gujarati (B). If the population is large and many units share the same $x$-values, then there will be many values of $y$ and $\varepsilon$ for units within a particular sub-population. Hence we can regard $\varepsilon_i$ as a random variable with probability density function $f$ defined by:

$$f(\varepsilon_i) = F_{\varepsilon}(\varepsilon | x = x_i)\quad\quad(1)$$

where $F$ is the joint probability density function of $x$ and $\varepsilon$. (I pass over the possibility that distributions might be discrete, which is somewhat tangential to the question.) Definition (1) allows that the probability density function of $\varepsilon$ may differ between sub-populations. Thus it can support the formulae for homoscedasticity and no autocorrelation, allowing that these properties may or may not be satisfied.

Turning now to the exogeneity assumption, Formulation (2) is inappropriate under either of these interpretations of the subscripts. If the subscript refers to a randomly selected unit, then the condition $\forall i$ implies that, for each particular unit, there is zero covariance between the values of $x$ and $\varepsilon$ for that unit. This makes no sense as those values are not random variables.

If on the other hand the subscript refers to a subpopulation defined by a particular value of $x$, we can allow as explained above that $\varepsilon_i$ is a random variable. But Formulation (2) still makes no sense because $x_i$ is not a random variable.

Formulation (1) is correct. The subscripts, though important for expressing the homoscedasticity and no autocorrelation assumptions, are not needed for expressing exogeneity. Without using the subscripts, Formulation (1) can be related to $F$, the joint probability density function of $x$ and $\epsilon$, as follows:

$$E[x] = \int_{-\infty}^{\infty} xF_x(x) dx \quad\quad(2)$$

$$E[\varepsilon] = \int_{-\infty}^{\infty} \varepsilon F_{\varepsilon}(\varepsilon) d\varepsilon \quad\quad(3)$$

$$E[x\varepsilon] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} x\varepsilon F(x,\varepsilon) d\varepsilon\ dx \quad\quad(4)$$

$$Cov(x,\varepsilon) = E[x\varepsilon] – E[x]E[\varepsilon] \quad\quad(5)$$

References

A. Wooldridge, J M (2nd edn 2010) Econometric Analysis of Cross Section and Panel Data p 8

B. Gujarati, D N (3rd edn 2006) Essentials of Econometrics p 136