1
$\begingroup$

I created a model for predicting a scalar variable from a set of features. I trained a linear regression on a training set, and used the resulting coefficients to produce predictions for a test set.

Then, I did simple linear regression to the predictions as a function of the ground truth values of the test set, expecting a slope of 1 and an intercept of 0. Although I got $R^2 \approx 1$, the slope was significantly different from 1 and the intercept with the vertical axis was significantly different from 0.

What does this tell me about the original linear regression?

What it be more informative to "force intercept to 0" for the second linear regression? This causes the intercept to be exactly 0, the slope becomes closer to 1 and the $R^2$ becomes somewhat smaller.

$\endgroup$
1
$\begingroup$

It just tells you that you won't have perfect fit in a linear regression on realistic datasets.

First linear regression will not fit the data completely, so there will be some unexplained variance remaining between predictions and original output value (on train as well as test set). Your second model will try to fit in that unexplained variance in the regression. So you are seeing non-one slope and non-zero intercept.

$\endgroup$
  • $\begingroup$ also, if you trained your data using an intercept, then you should estimate the simulated data using an intercept also. Generally , unless the predictor is gasoline amount started with and the response is miles obtained before gas runs out, an intercept is usually preferred. $\endgroup$ – mlofton Jul 6 at 19:45
0
$\begingroup$

You mention that you performed a linear regression on the residual plot, which yielded $R^2\approx1$. What this tells you is actually very significant: there is a pattern in the residuals. And if there is a pattern in the residuals, that means your regression did not adequately capture the trend of your data. Now you know that the relationship between your explanatory and response variable is not linear.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.