When to use non-parametric regression?

Question

I am using PROC GLM in SAS to fit a regression equation of the following form

$$ Y = b_0 + b_1X_1 + b_2X_2 + b_3X_3 + b_4t $$

The QQ plot of the resulting redsiduals indicate deviation from normality. Any transformation of $Y$ is not useful in making the residuals normal.

At this point, can I safely switch to non-parametric methods such as PROC LOESS.

I have already used PROC LOESS, and the fit looks better than PROC GLM. But I do not have much knowledge in non-parametric regression. I do not know when to choose non-parametric regression over parametric regression.

Can someone help me with this?

I will go ahead and add another question. Following are the description of my variables in the model. $$ Y =\text{cost of medical care}\\ X_1 =\text{number of injections}\\ X_2 =\text{number of surgeries}\\ X_3 =\text{number of physical therapies}\\ t =\text{time} $$ Sometimes I get negative predicted cost. This does not make sense. How can I address this issue?

Answer 1

Before looking on QQplots of residuals, you should assess the quality of fit, by plotting residuals against the predictors in the model (and possibly, also against other variables you have which you did not use). Non-linearity should show up in this plots. If the effect of variable $x$ really is linear, you expect the plot of residuals against $x$ to be "horizontal", without visible structure:

                                                                   *
*                 *
      *                               *
        *   
                                                  *
--------------------------------------*------------------------------x
   *     
           *

                                     *
       *                                                    *
                                *

That is, a random horizontal "blob" of points, centered around the line resid=0.

If the effect is non-linear, you expect to see some curvature in this plot. (and, please, ignore the QQplots until you got non-linearities sorted out, using plots as above!)

You should also think about possible interactions (modelled usually by product terms), that is, the effect of one variable depends on the levels of another, (If all your three variables have high values at the same time, maybe that shows some particularly difficult patient? If so, interactions could be needed).

If you go for some non-linear model, after having tried for interactions and transformations (did you try log(Cost)?) Did you try some box-cox-transformations? Since you have multiple regression, I don't think that loess is what you need, you should look for gam (generalized additive models, SAS should have that, in R it is in package mgcv).

Answer 2

A LOESS will always give a better fit than regression, unless the data truly lie along a straight line. LOESS is a locally linear approximation that is designed to pass close to the data. These methods are basically exploratory. And while it is dangerous to extrapolate a linear model beyond the limits of the fit, extrapolation would be reckless in the case of LOESS.

If your model gives you negative costs, that's a pretty good sign that a linear regression is not appropriate on the variables you have. You say that you tried transformations. Did you take the log of cost against your predictors?

In the nature of things, it is unlikely that there is a simple relationship between cost and the variables you mention. Sometimes the purpose of a linear regression is simply to demonstrate that some sort of correlation exists, and perhaps to select a sensible set of predictors.

Answer 3

Bravo for doing residual analysis. Puts you way ahead of the typical analyst. (Your description of the model is deficient in not describing the error structure, though.) You should be considering transformations of the X's as well as looking at transformations of the Y's. I realize that SAS is behind R in modeling with spline fits but I understand that recent versions have offered that capacity. Consider adding restricted cubic spline fits for the X terms. As a reference Frank Harrell's text "Regression Modeling Strategies" is hard to beat. It has solid statistical arguments for this approach. It is a parametric approach that allows discovery of structure in the data that would other wise be missed.

Answer 4

I think kjetil has given you some good suggestions. I would add that non-normal residuals does not mean you have to jump from linear or nonlinear regression to nonparametric regression. By going to nonparametric regression you give up the structure of a functional form. There are robust regression alternative to OLS regression that you could go to first. Then generalized linear models and generalized additive models if next steps are needed. LOESS should in my view be your last resort. I think that I agree with kjetil on that.