Is sample homogeneity an assumption of regression analysis?

Question

I have assumed (i.e. I think I was taught, longer ago than I can remember) that regression analyses assume that a sample is homogeneous. If it is not, then the appropriate thing to do is either add dummy variables to code for the different groups included in the sample, or carry out an ANCOVA to test whether group parameters are equal. Does ignoring the heterogeneity of a sample invalidate a regression analysis?

Answer 1

The sample is typically assumed to be homogeneous in the sense that the error terms $\epsilon_i$ in the equation $y_i=\beta_0+\beta_1x_1+\beta_2x_2+\ldots+\epsilon_i$ satisify the following conditions:

1. All have mean zero: $\rm E(\epsilon_i)=0$ for all $i$,
2. Are uncorrelationed: $\rm Cov(\epsilon_i,\epsilon_j)=0$ for $i\neq j$,
3. All have the same variance: $\rm Cov(\epsilon_i)=\sigma^2$ for all $i$.

These are known as the Gauss-Markov conditions and ensures that the ordinary least squares estimator performs well (unbiasedness, best linear unbiased estimator...).

Note that these conditions can be satisfied even if you have observations from different groups. Oftentimes, that is however not the case. If there are differences in mean between the groups, the first and second conditions are violated. If there are correlations within the groups, the second condition is violated. If the groups differ in variance, the third is violated.

Violation of the Gauss-Markov conditions can cause all sorts of problems. For some of the consequences of non-constant variance, see the Wikipedia page on heteroscedasticity.

Transformations can be useful when the third condition isn't met, but if the different groups cause problems with conditions one and two, it seems more reasonable to add a group dummy variable or to use ANCOVA.