# Is sample homogeneity an assumption of regression analysis?

I have assumed (i.e. I think I was taught, longer ago than I can remember) that regression analyses assume that a sample is homogeneous. If it is not, then the appropriate thing to do is either add dummy variables to code for the different groups included in the sample, or carry out an ANCOVA to test whether group parameters are equal. Does ignoring the heterogeneity of a sample invalidate a regression analysis?

-

The sample is typically assumed to be homogeneous in the sense that the error terms $\epsilon_i$ in the equation $y_i=\beta_0+\beta_1x_1+\beta_2x_2+\ldots+\epsilon_i$ satisify the following conditions:

1. All have mean zero: $\rm E(\epsilon_i)=0$ for all $i$,
2. Are uncorrelationed: $\rm Cov(\epsilon_i,\epsilon_j)=0$ for $i\neq j$,
3. All have the same variance: $\rm Cov(\epsilon_i)=\sigma^2$ for all $i$.

These are known as the Gauss-Markov conditions and ensures that the ordinary least squares estimator performs well (unbiasedness, best linear unbiased estimator...).

Note that these conditions can be satisfied even if you have observations from different groups. Oftentimes, that is however not the case. If there are differences in mean between the groups, the first and second conditions are violated. If there are correlations within the groups, the second condition is violated. If the groups differ in variance, the third is violated.

Violation of the Gauss-Markov conditions can cause all sorts of problems. For some of the consequences of non-constant variance, see the Wikipedia page on heteroscedasticity.

Transformations can be useful when the third condition isn't met, but if the different groups cause problems with conditions one and two, it seems more reasonable to add a group dummy variable or to use ANCOVA.

-
+1. If there are differences between the group means and it is ignored in the model fitting, then the model will fit the best approximation (within the subspace that forces the groups to be homogeneous) which still has $E(\varepsilon)=0$, which will effectively average the coefficients over the groups. Of course, when the group means are different, this model fit isn't terribly useful, unless you're trying to make inference about a randomly selected person whose group membership you don't know. – Macro Jul 4 '12 at 13:49
I am removing my answer not because anything is wrong but rather because the latter answer by MansT covers it more completely except for the part about modeling the variance function as described in Ray Carroll's book. – Michael Chernick Jul 4 '12 at 13:57