# When to use non-parametric regression?

I am using PROC GLM in SAS to fit a regression equation of the following form

$$Y = b_0 + b_1X_1 + b_2X_2 + b_3X_3 + b_4t$$

The QQ plot of the resulting redsiduals indicate deviation from normality. Any transformation of $Y$ is not useful in making the residuals normal.

At this point, can I safely switch to non-parametric methods such as PROC LOESS.

I have already used PROC LOESS, and the fit looks better than PROC GLM. But I do not have much knowledge in non-parametric regression. I do not know when to choose non-parametric regression over parametric regression.

Can someone help me with this?

I will go ahead and add another question. Following are the description of my variables in the model. $$Y =\text{cost of medical care}\\ X_1 =\text{number of injections}\\ X_2 =\text{number of surgeries}\\ X_3 =\text{number of physical therapies}\\ t =\text{time}$$ Sometimes I get negative predicted cost. This does not make sense. How can I address this issue?

• You can of course avoid predicting negative costs by modelling the log of it: $log(Y) = b_0 + b_1X_1 + b_2X_2 + b_3X_3 + b_4t$ – Dirk Horsten Aug 7 '15 at 10:17

Before looking on QQplots of residuals, you should assess the quality of fit, by plotting residuals against the predictors in the model (and possibly, also against other variables you have which you did not use). Non-linearity should show up in this plots. If the effect of variable $x$ really is linear, you expect the plot of residuals against $x$ to be "horizontal", without visible structure:

                                                                   *
*                 *
*                               *
*
*
--------------------------------------*------------------------------x
*
*

*
*                                                    *
*


That is, a random horizontal "blob" of points, centered around the line resid=0.

If the effect is non-linear, you expect to see some curvature in this plot. (and, please, ignore the QQplots until you got non-linearities sorted out, using plots as above!)

You should also think about possible interactions (modelled usually by product terms), that is, the effect of one variable depends on the levels of another, (If all your three variables have high values at the same time, maybe that shows some particularly difficult patient? If so, interactions could be needed).

If you go for some non-linear model, after having tried for interactions and transformations (did you try log(Cost)?) Did you try some box-cox-transformations? Since you have multiple regression, I don't think that loess is what you need, you should look for gam (generalized additive models, SAS should have that, in R it is in package mgcv).

• Thanks for the valuable information. I did try the log(cost) with the linear regression, but it did not help much. I will add the interaction effects and observe what happens. I will also experiment with transformations again. I will keep everyone posted with my developments and findings. – ann Sep 20 '12 at 2:25

A LOESS will always give a better fit than regression, unless the data truly lie along a straight line. LOESS is a locally linear approximation that is designed to pass close to the data. These methods are basically exploratory. And while it is dangerous to extrapolate a linear model beyond the limits of the fit, extrapolation would be reckless in the case of LOESS.

If your model gives you negative costs, that's a pretty good sign that a linear regression is not appropriate on the variables you have. You say that you tried transformations. Did you take the log of cost against your predictors?

In the nature of things, it is unlikely that there is a simple relationship between cost and the variables you mention. Sometimes the purpose of a linear regression is simply to demonstrate that some sort of correlation exists, and perhaps to select a sensible set of predictors.

• It makes so much sense when u mentioned that negative costs indicates that linear regression might not be appropriate. I will continue my analysis and add some interactions. Thank you. – ann Sep 20 '12 at 2:31

Bravo for doing residual analysis. Puts you way ahead of the typical analyst. (Your description of the model is deficient in not describing the error structure, though.) You should be considering transformations of the X's as well as looking at transformations of the Y's. I realize that SAS is behind R in modeling with spline fits but I understand that recent versions have offered that capacity. Consider adding restricted cubic spline fits for the X terms. As a reference Frank Harrell's text "Regression Modeling Strategies" is hard to beat. It has solid statistical arguments for this approach. It is a parametric approach that allows discovery of structure in the data that would other wise be missed.

• Thanks for the appreciation DWin. I just graduated, and this is my first job as an analyst. Coincidentally, this kind of analysis is new to the company too. So, I am just trying to come up an analysis that is not completely nonsense. I will take your suggestion and try transformations on both $Y$ and $X$ variables. I will also go through the reference. I just found a pdf version online. Thanks for your input. – ann Sep 20 '12 at 3:03
• There is something wrong with log transforming in and outputs that intuitively depend linearly. If you model $log(Y) = b_0 + b_1log(X_1) + b_2log(X_2)$ where $X_1$ are injections in the right arm and $X_2$ are injections in the left arm, you predict completely different costs for someone having all injections in the same arm and someone who having half of them at each side – Dirk Horsten Aug 7 '15 at 10:36
• Your comment seems rather tangential to my response (and to the question since partitioning of injections by arm was never mentioned) I hope you don't think that spline functions are equivalent to log transformations. The log transformation of the Y creates a model where the model is multiplicative in the predictors when transformed back to the cost scale. That's a fairly big change and one whose problems you have not adequately describe to the questioner. – DWin Aug 7 '15 at 16:52

I think kjetil has given you some good suggestions. I would add that non-normal residuals does not mean you have to jump from linear or nonlinear regression to nonparametric regression. By going to nonparametric regression you give up the structure of a functional form. There are robust regression alternative to OLS regression that you could go to first. Then generalized linear models and generalized additive models if next steps are needed. LOESS should in my view be your last resort. I think that I agree with kjetil on that.