1
$\begingroup$

I read somewhere,

In a regression problem, if the relationship between each predictor variable and the criterion variable is nonlinear, then the predictions may systematically overestimate the actual values for one range of values on a predictor variable and underestimate them for another.

Does anyone have an explanation to this behavior?

$\endgroup$
  • 2
    $\begingroup$ nonlinearity is the explanation, assuming you used a linear regression problem. Just draw some non-linear curve approximating some straight line. $\endgroup$ – kjetil b halvorsen Nov 3 '15 at 18:39
  • $\begingroup$ is there a way to customize this range of values? $\endgroup$ – Farhaneh Moradi Nov 3 '15 at 19:19
  • $\begingroup$ Why do you want your algorithm to predict some wrong value? $\endgroup$ – kjetil b halvorsen Nov 3 '15 at 19:22
  • 1
    $\begingroup$ consider, I have a dataset that it's target values range from -1 to 1. For test cases with target value near 1, I want to get predicted value larger than it's real value in test set. For example if the real value in a test case is 0.9, I want from my regression algorithm to predict it 0.95. And for test cases with target value near -1, I want to get predicted value smaller than it's real value in test set. For example if the real value in a test case is -0.9, I want to my regression algorithm to predict it -0.95. Is it possible? $\endgroup$ – Farhaneh Moradi Nov 3 '15 at 19:48
  • 1
    $\begingroup$ The question seems perfectly clear to me. I don't see that the OP wants to do this; they just seem to want to understand it, which I think is perfectly reasonable. $\endgroup$ – gung Nov 4 '15 at 0:49
3
$\begingroup$

Here is a simple example (coded in R). Hopefully the image is sufficiently obvious to explain how a nonlinear relationship (not model), when fit with a linear relationship, yields regions with predicted values that are systematically overestimated and underestimated.

set.seed(7439)                # this makes the example exactly reproducible
x = runif(50, min=0, max=15)  # X is uniformly distributed from 0 to 15
 # this is the true data generating process: 
y = 3.7 - 2.5*x + 0.56*x^2 - 0.028*x^3 + rnorm(50, mean=0, sd=.3)
model = lm(y~x)               # here I fit a model with a linear relationship

windows()
  plot(x, y)                       # this plots the data
  abline(coef(model), col="gray")  # this plots the model's predicted values

enter image description here

$\endgroup$
  • $\begingroup$ thanks, but what happened when we have nonlinear relationship and also model it with a nonlinear algorithm? $\endgroup$ – Farhaneh Moradi Nov 4 '15 at 5:11
  • 1
    $\begingroup$ If the model's functional from is appropriate, you should be fine. $\endgroup$ – gung Nov 4 '15 at 5:29
  • $\begingroup$ thanks, but what happened when we have nonlinear relationship and also model it with a nonlinear algorithm? I saw overestimated and underestimated predictions in low and high values of targets respectively when using nonlinear model too. $\endgroup$ – Farhaneh Moradi Nov 4 '15 at 5:32
  • $\begingroup$ This was just answered by @gung. If you pick the correct functional form of the nonlinear model, you should be fine. There shouldn't be a systematic underestimate or overestimate. $\endgroup$ – StatsStudent Nov 4 '15 at 5:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.