In logistic regression there is Box-Tidwell but I know of nothing like that in linear regression. I use partial residual plots to look for this, a graphical feature, but would love to find a formal test (in honesty I doubt you can do a formal test of this, but I could be wrong).
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Related: stats.stackexchange.com/questions/70009/…. $\endgroup$– StubbornAtomCommented May 1, 2019 at 10:31
-
$\begingroup$ For the model $y=\beta_0+\sum_j\beta_jx_j+\varepsilon$, isn't a formal test $H_0:\beta_j=0$ for all $j$ vs $H_1: \text{not }H_0$? This is similar to an ANOVA F-test. $\endgroup$– StubbornAtomCommented May 1, 2019 at 10:41
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
5
Box-Tidwell was developed for ordinary least squares regression models.
So if you were inclined to use Box-Tidwell for this, that's actually what it's designed for.
It's not the only possible approach, but it sounds like an approach you're already familiar with.
However, I'm not convinced that (most times it's used) a formal test is appropriate - I believe it usually answers the wrong question, while the diagnostic plots you've been looking at come closer to answering a useful question. [I have a similar opinion of many other tests of regression assumptions]
-
4$\begingroup$ Would I be better off with a different specification is usually a good question, but one hard to tackle except very specifically. (Pun not really intended, but it seems to fit the case.) $\endgroup$– Nick CoxCommented May 1, 2019 at 7:00
-
$\begingroup$ @Glen_b Could you state the "wrong" and "useful" questions you refer to? Thanks. $\endgroup$– rolando2Commented May 1, 2019 at 12:28
-
4$\begingroup$ The hypothesis being tested is exact linearity -- which will almost never be the case. If we fail to reject, all we learned was that our sample was too small to detect the nonlinearity, not that its effect was small. If we do reject, we're no better off, we learned what we already knew, but if the nonlinearity is small, it is of little consequence. The test still doesn't tell us whether the nonlinearity actually matters; what we needed to know is how much difference the nonlinearity we have makes to our inference. $\endgroup$– Glen_bCommented May 1, 2019 at 12:48
-
$\begingroup$ A problem I have, because I work with the entire population in question normally, is that I have thousands of data points. They tend to look like big blobs in the residuals so its hard to discern patterns in the regression; they don't represent very well what you see in text.books. $\endgroup$ Commented May 1, 2019 at 20:40
-
1$\begingroup$ 1. With a large sample, that would form the basis of a (readily answered) question (how to see nonlinearity in a residual plot in such circumstances); ideally adding an example plot that would give you difficulty. 2. If you're fitting to the entire population of interest, notions of testing go out the window (you certainly don't have a random sample!). You literally have the whole thing you want to make inferences about, just calculate what you need. $\endgroup$– Glen_bCommented May 2, 2019 at 0:07
$\begingroup$
$\endgroup$
1
The best formal tests come from relaxing the linearity assumption, then seeing if removing the nonlinearities damages the explained variation in Y. For example you can expand X using a regression spline and test the nonlinear components. My RMS Course Notes goes into details.
But once you've allowed for the possibility of nonlinearity, you distort statistical inference by removing the nonlinear terms. The real numerator degrees of freedom for the regression are the number of chances to give the model, which must take into account the nonlinear terms. So the best advice overall is to allow effects not known to be linear to be be nonlinear and be done with it. This will preserve confidence interval coverage, etc.
answered May 1, 2019 at 11:01
-
$\begingroup$ "So the best advice overall is to allow effects not known to be linear to be be nonlinear and be done with it. This will preserve confidence interval coverage, etc." I am not sure how you do that, but in any case in the areas I work in there is no well developed theory (very little in the way of sophisticated statistical analysis at all). $\endgroup$ Commented May 1, 2019 at 20:37
$\begingroup$
$\endgroup$
1
Fit a non-linear regression (e.g. spline model like GAM) and then compare it to the linear model using AIC or likelihood ratio test. This is a simple and intuitive method of testing non-linearity. If the test rejects, or if AIC prefers the GAM, then conclude there are non-linearities.
-
1$\begingroup$ That is an interesting suggestion gammodel, but I have a question. I have many, say 30, predictors in my model. The AIC is going to tell you about the overall model. How would I know which of the individual variables in the model was actually non-linear? $\endgroup$ Commented May 7, 2019 at 20:47