0
$\begingroup$

Two assumptions of linear regression are:

  1. Linearity: $E[Y_i|X_i] = \beta^{*t}X_i = E[\epsilon_i] = 0$

  2. The variance is fixed and it isn't dependent on $i$. Or in other words, homoscedasticity.

The simplest check is to plot the residuals ($e_i = Y_i - \hat{Y_i}$) as the Y-axis and the predicted values as the X-axis.

We expect:

  1. From the linearity assumption: the residuals should be around the line e = 0.

  2. From the constant variance assumption: similar spread of the residuals around the line = 0.

I've made such a plot. I found the predictions and the residuals with this code: (X are the predictive features and Y is the target feature).

predictions = X @ np.linalg.lstsq(X, Y, rcond=None)[0]
residuals = Y - predictions

And made the plot to check the two assumptions above, but I don't know how to interpret this plot. Does this mean that these two assumptions are reasonable? what can I infer about the spread of the residuals?

enter image description here

EDIT -

I've also made a Q-Q plot using the residuals in order to check the normality of the noise, using this code:

stats.probplot(residuals, dist="norm", plot= plt)
plt.title("MODEL Residuals Q-Q Plot")
plt.legend(['Actual','Theoretical']);

I got this:

enter image description here

I added this edit because I don't know if this has to do with the first plot. If it helps to interpret the first plot then here it is. Thanks!

$\endgroup$
9
  • 4
    $\begingroup$ On striping of residuals, see my answer at stats.stackexchange.com/questions/138908/… $\endgroup$
    – Nick Cox
    Dec 24, 2022 at 12:29
  • 3
    $\begingroup$ It's a minority view, but here is mine: We would be much better off if we talked much more about ideal conditions, and much less about assumptions. $\endgroup$
    – Nick Cox
    Dec 24, 2022 at 12:33
  • 1
    $\begingroup$ This is undoubtedly a great plot to see. In your case, it raises a desire to see any plot of the distribution of $Y$ that shows all possible detail, e.g. a stacked or jittered dot or strip plot or a quantile plot (emphatically not a box plot or a histogram). $\endgroup$
    – Nick Cox
    Dec 24, 2022 at 12:37
  • 1
    $\begingroup$ To check for homoscedasticity directly, plot and smooth absolute residuals as a function of fitted. There are many twists on that as the simplest recipe. In practice the assumptions are almost never well satisfied outside simulations, as better textbooks explain. I would call linearity not so much an assumption as an epitome of what you are doing, fitting a linear function. $\endgroup$
    – Nick Cox
    Dec 24, 2022 at 13:05
  • 2
    $\begingroup$ The outer boundary lines show bounds, I guess 0 to 100. Just look a histogram of your Y data; this estimates your marginal distribution. Now, regression is simply a model for how that distribution changes when you fix the X's at different values, i.e., it is a model for the conditional distributions of Y. The beta family for those conditional distributions of Y might provide a good model for these distributions. No need to bring residuals into it, as they just provide a misdirection. $\endgroup$ Dec 24, 2022 at 16:46

1 Answer 1

0
$\begingroup$

Your plot shows that large predictions are on average too high (most residuals are negative) and small predictions are on average too small. So the assumption of linearity is proved false.

In such a case, to build a good quality model with a linear regression, you first need a non-linear transformation of the Y value. Is there any reason to expect that the underlying process is multiplicative rather than additive? If so, trying to predict $log(Y)$ is reasonable. More rarely, e.g. $\sqrt{Y}$ might be appropriate.

$\endgroup$
1
  • $\begingroup$ The tilt pattern makes it too easy to infer bias where there is none. The residual plot may imply nonlinearity but the mean residual is guaranteed to be zero. $\endgroup$
    – Nick Cox
    Dec 25, 2022 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.