4
$\begingroup$

I have performed a simple regression analysis between one dependent variable (DV) and one explanatory variable (IV).

If the p-value from the regression analysis for the IV is not significant, should I still use residual plots to verify that the regression model used was correct (and the statement of non-significant IV is correct)?

Or should one use residual plots only for models that include significant IVs?

$\endgroup$
  • $\begingroup$ non-significant coefficients still mean that a model exists. the model simply states that the dependent variables is just equal to the error term. hence all the model building procedures should ideally be followed. also in view of a larger picture, generally to document the results, all the procedures should be done so that if at a later stage you test the same set of variables and a dependency comes out, you can make a comparative study. $\endgroup$ – htrahdis Oct 24 '13 at 13:55
7
$\begingroup$

They are still useful in assessing whether the relationship between the explanatory variables and the dependent variable is linear (or modeled properly given the equation). For an extreme example, I generated some data with a quadratic relationship and fit a linear regression of the form $Y = \alpha + \beta(X) + e$. (Because the parabola is approximately centered on zero $\beta$ is insignificant in the equation).

enter image description here

If you plot $X$ versus the residuals though the quadratic relationship is still very clear. (Imagine just detilting the first plot.)

enter image description here

I'm sure you can dream up other scenarios in which regression coefficients are insignificant but examining the residuals will show how the model is inadequate.

$\endgroup$
  • $\begingroup$ OK thanks. I plotted a residual plot that looks pretty good, see postimg.org/image/gz4twyytd. So the residual plot says the model is properly given even the fact that the IV is not significant. $\endgroup$ – eXpander Oct 24 '13 at 13:59
  • $\begingroup$ So I guess you can verify a non-significant result by having a good residual plot showing the model is correctly defined. $\endgroup$ – eXpander Oct 24 '13 at 14:03
  • $\begingroup$ That plot shows that there is no heteroscedasticity @user1261558. I suggest a different plot though, of the residuals on the Y axis and the explanatory variable on the X axis. This is just an example though of where residual plots would be useful even with a variable that was not significant in a regression equation. $\endgroup$ – Andy W Oct 24 '13 at 14:03
  • $\begingroup$ That is the explanatory variable, on the X axis (standardized). Thanks! $\endgroup$ – eXpander Oct 24 '13 at 14:05
  • $\begingroup$ Really? Regression Standardized Predicted Value is the label for the one explanatory variable - seems a bit strange if not misleading. (FYI I'm pretty sure you have enough rep now to post images to your question.) $\endgroup$ – Andy W Oct 24 '13 at 14:08
4
$\begingroup$

Assume for simplicity that you have fitted some line $\hat y = b_0 + b_1 x$ given a dependent or response variable $y$ and a predictor or independent variable $x$. This specific assumption can be relaxed, which we will get to in good time.

With one variable on each side, a residual plot (meaning, a plot of residual $y - \hat y =: e$ versus fitted or predicted $\hat y$) in principle shows just the same information as a scatter plot with regression line superimposed. On the latter, the residuals are just the vertical differences between the data points and the line and the fitted are the corresponding values on the line, i.e. for the same value of $x$.

In practice, a residual plot can make structure in the residuals more evident:

  • The regression line is rotated to the horizontal. Seeing structure in anything is easiest when the reference indicating no structure is a horizontal straight line, here the line $e = 0$.

  • There is better use of space.

In this easy example, some structure in the residuals is discernible in the scatter plot

enter image description here

but even easier to see in the residual plot:

enter image description here

The recipe here was simple. The data were fabricated as a quadratic plus Gaussian noise, but the quadratic is only roughly captured by the naive linear fit.

But it is still generally true that structure is easier to see on a residual plot. Some caution is needed in not over-interpreting residual plots, especially with very small sample sizes. As usual, what you spot should make scientific or practical sense too.

What if the fitted is more complicated than $b_0 + b_1 x$? There are two cases:

  • Everything can still be shown on a scatter plot, e.g. the right-hand side is a polynomial or something in trigonometric functions of $x$. Here, if anything, the residual plot is even more valuable in mapping everything so that zero residual is a reference.

  • The model uses two or more predictors. Here also the residual plot can be invaluable as a kind of health check showing how well you did and what you missed.

The health check analogy is a fair one more generally: Residual plots can help you spot if something is wrong. If nothing is evidently wrong, no news is good news, but there is no absolute guarantee: something important may have been missed.

On whether the predictor had a significant effect, I know of no rule whatever for drawing or not drawing a residual plot. In the concocted example here, significance levels and figures of merit such as $R^2$ are extremely good, but the straight line model still misses a key part of the real structure. Conversely, a residual plot often illuminates why a model failed to work: either the pattern really is all noise, so far as can be seen, or your model misses something really important, such as some nonlinearity.

Footnote: for many statistical people IV means instrumental variable, not independent variable.

$\endgroup$
  • $\begingroup$ My writing this clearly overlapped with @Andy W's writing.... $\endgroup$ – Nick Cox Oct 24 '13 at 14:17
  • $\begingroup$ This is better than mine! This shows with one variable linear regression that plotting residual ~ predicted and residual ~ explanatory makes little difference (at least I can't think of any scenarios in which it makes a difference). $\endgroup$ – Andy W Oct 24 '13 at 14:23
  • $\begingroup$ Thanks! I think if you start fitting e.g. exponential trend + sinusoidal seasonal trend with changing amplitude as model, anye structure in residuals gets harder to see on the original scatter plot. The scatter plot plus curve and the residual plot are complementary; if it does happen that one does not add much information to the other, it is painless to produce it, look at it quickly and discard it (or should be with decent software). $\endgroup$ – Nick Cox Oct 24 '13 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.