I am using linear regression in R to predict the target using a single predictor. I observe reduced range and reduced variable of the predicted values compared to the original target.

xy = read.csv("test_xy.csv", header=T)
model = lm(y ~ x, xy)

#lm(formula = y ~ x, data = xy)
#(Intercept)            x  
#     0.1276       0.3551  

#   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#-2.2372 -0.4076  0.1567  0.1363  0.7556  2.0358 

#[1] 0.8827763

# now predict and look at the summary 
yy = predict(model, xy)

#    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
#-0.76281 -0.01873  0.13813  0.13631  0.34179  0.97745 

#[1] 0.3436133

The variables x and y are moderately correlated and have similar summary statistics.

#    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
#-2.50746 -0.41212  0.02962  0.02449  0.60314  2.39317 

#[1] 0.967628

cor(xy$x, xy$y)
#[1] 0.3892417 

Why does this happen?

The data is available here.

  • $\begingroup$ hmmm... isn't this what should happen if your fit is meaningful? Imagine you have a predicted slope, and all your predicted points are now on the slope.. they will definitely closer to theirs means, as compared to the actual data, which can float anywhere $\endgroup$ – StupidWolf Jun 3 at 16:34
  • 1
    $\begingroup$ Probably you should first understand what linear regression is. Please take your time to understand it. Otherwise any elaborations made will not be of any help, since we will assume you already understand the subject, and the relationship between the coefficient beta and the correlation $\endgroup$ – Onyambu Jun 3 at 16:45
  • $\begingroup$ thanks @StupidWolf and @Onyambu, yes it does make sense that the data gets projected on a linear line and hence gets closer to the mean. However, what I am looking for is whether this reduction is related to the error term and if yes then how? Also as a wider consequence, does this mean that in a predictive setting we can always expect somewhat squashed predictions? $\endgroup$ – DataD'oh Jun 3 at 16:52
  • 1
    $\begingroup$ Hey @DataD'oh, by error i suppose you are referring to y = ax + b + error. When you fit a linear model, you try to minimize the error, so yeah for sure at the end of the day, the variance or sd is lower $\endgroup$ – StupidWolf Jun 3 at 16:59
  • $\begingroup$ You can have a look at this article: the total sum of squares is always (mathematically) greater or equal to the explained sum of squares (since TSS = ESS + RSS: Total Sum of Squares = Explained Sum of Squares + Residual Sum of Squares) $\endgroup$ – Valeri Voev Jun 3 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.