Why in ordinary linear regression is no global test for lack of model fit unless there are replicate observations at various settings of X?

I read this quote from Regression Modelling Strategies.

In ordinary linear regression there is no global test for lack of model fit unless there are replicate observations at various settings of X. This is because ordinary regression entails estimation of a separate variance parameter $$\sigma^2$$

I assume the first part of the sentence is because there is no way to separate the individual error from the model (not sure if this is correct), but I don't understand why the second sentence gives an explanation to the first.

Further, if my assumption is true, shouldn't this problem exist for non-linear regression; e.g., logistic regression?

Just a side question as well, I may not be 100% clear on the exact definition of a global test. I've many references to it but I couldn't find a definition in any of the textbooks I have.

1 Answer

As the writer of the second sentence, I’m not positive what I meant.

It’s good to distinguish omnibus assessments of lack of fit from directed assessments. With replicates (duplicated X combinations) you can get a measure of “pure irreducible error” and compare that to regular residual variation with OLS (omnibus approach). But in general we use directed assessments, and making judgements about goodness of fit is better thought of as asking the question of whether relaxing assumptions leads to better prediction as judged by indexes such as AIC or $$R^{2}_\text{adj}$$. Relaxing assumptions can entail splining all continuous predictors and adding a bunch of 2-way interaction terms, for example, then getting “chunk test” type of assessments.

• Thanks for your response. I just wanted to clarify, without replicates, is it also not possible to do the global test for lack of fit for nonlinear models? Also, if there are, how would it look (I don't know how decomposing the error would look with a binary dependent variable)? Commented Jun 20 at 15:01
• Now I’m thinking my second sentence made more sense. Without a residual variance, a logistic model can’t have a global omnibus test unless one can make more sense of “residual deviance” than I can. So we are left with formulating specific model departures such as non linearity and interaction. These are advantageous anyway because they are actionable. Another philosophy I espouse in RMS is to pre-specify a model that is as flexible as the sample size allows. If that model doesn’t fit there may be nothing you can do about it anyway. Commented Jun 21 at 11:13
• Ah I see, the second sentence was referring to just the second clause in the first sentence, and it is as opposed to nonlinear models. Commented Jun 21 at 12:43