Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am fitting a linear model to a problem, and a little confused by what is going on. Without the details here are the two plots confusing me:

Residuals vs Fitted

Residuals vs Fitted

Residuals vs Y

Residuals vs Y

Now the residuals vs fitted looks good to me. Fairly evenly dispersed, no clear pattern. However, the $y$ vs fitted do not look good. I would have expected there to also not be a clear trend in this relationship.

This seems basic, but I'm rather lost. I don't want to fit a model that allows large positive errors at high values of $Y$ to make up for large negative errors at low values of $Y$.

Is this just an indication of a poorly fitting model? Or is something else going on here?

EDIT: Asked for fitted vs y (changed to have labels)

fitted vs y

EDIT2: Just want to point out that this question has already been asked and answered (albeit in a more abstract sense) What is the expected correlation between residual and the dependent variable? .

share|improve this question
    
What does your fitted v/s y look like? –  Arun Jose May 6 at 6:05
    
Added to the question. –  thebigdog May 6 at 6:15

1 Answer 1

up vote 3 down vote accepted

1) The residuals and the fitted are uncorrelated by construction. In fact if there was any correlation between them, there would be uncaptured linear trend in the data - we could get a closer fit by changing the coefficients until they were uncorrelated.

2) The residuals and the y-variable are always positively correlated. This is a necessary consequence of (1).

$$cov(e,y) = cov(e,e+\hat y) = \sigma^2+0 = \sigma^2$$

So it would be surprising if there wasn't a trend in that first plot.


Consider a simulated example -

Note that by plotting residuals against observed, it's equivalent to using slanted axes in the residuals vs fitted plot:

enter image description here

The reason for the high observed (the grey slanted lines mark constant observed, the ones to the far right are high) being associated with high residuals is clear here, as it the reason for them being only positive near the end.

share|improve this answer
    
Well.. that was simple. Thanks! –  thebigdog May 6 at 6:44
    
I'm not able to reconcile one simple fact. Residuals would be positive and negative for all predicted values. How then, does the residual for higher y values end up being only positive and lower for small values of y. (More from intuition, than fact) –  Arun Jose May 6 at 9:43
    
Arun, I'm also still a little confused. But perhaps think of it this way: Large values of y are 'more likely' to correspond to data points lying above the regression line, while the opposite is true for negative values.. It sure isn't intuitive though. –  thebigdog May 6 at 10:44
1  
I feel bad for having missed this earlier, but there are a lot of good answers here: stats.stackexchange.com/questions/5235/… –  thebigdog May 6 at 11:03
    
Would just like to thank Glen_b for adding the extra information to his answer - definitely helping me develop an intuition for the situation. –  thebigdog May 6 at 21:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.