Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have run a simple linear regression of the natural log of 2 variables to determine if they correlate. My output is this:

R^2 = 0.0893

slope = 0.851

p < 0.001

I am confused. Looking at the $R^2$ value, I would say that the two variables are not correlated, since it is so close to $0$. However, the slope of the regression line is almost $1$ (despite looking as though it's almost horizontal in the plot), and the p-value indicates that the regression is highly significant.

Does this mean that the two variables are highly correlated? If so, what does the $R^2$ value indicate?

I should add that the Durbin-Watson statistic was tested in my software, and did not reject the null hypothesis (it equalled $1.357$). I thought that this tested for independence between the $2$ variables. In this case, I would expect the variables to be dependent, since they are $2$ measurements of an individual bird. I'm doing this regression as part of a published method to determine body condition of an individual, so I assumed that using a regression in this way made sense. However, given these outputs, I'm thinking that maybe for these birds, this method isn't suitable. Does this seem a reasonable conclusion?

share|improve this question
1  
The Durbin-Watson statistic is a test for serial correlation: that is, to see whether adjacent error terms are mutually correlated. It says nothing about the correlation between your X and your Y! Failing the test is an indication that the slope and p-value should be interpreted with caution. –  whuber Jul 19 '11 at 21:29
    
Ah, ok. That makes a little more sense than whether the two variables themselves are correlated...after all, I thought that's what I was trying to find using the regression. And that failing the test indicates I should be cautious interpreting the slope and p-value makes even more sense in this case! Thanks @whuber! –  Mog Jul 20 '11 at 0:34
    
I would just like to add a slope can be very significant (p-value<.001) even though the relationship is weak, especially with a large sample size. This was hinted at in most of the answers as that the slope (even if it's significant) says nothing about the strength of the relationship. –  Glen Jul 20 '11 at 12:30
add comment

3 Answers

up vote 17 down vote accepted

The estimated value of the slope does not, by itself, tell you the strength of the relationship. The strength of the relationship depends on the size of the error variance, and the range of the predictor. Also, a significant $p$-value doesn't tell you necessarily that there is a strong relationship; the $p$-value is simply testing whether the slope is exactly 0. For a sufficiently large sample size, even small departures from that hypothesis (e.g. ones not of practical importance) will yield a significant $p$-value.

Of the three quantities you presented, $R^2$, the coefficient of determination, gives the greatest indication of the strength of the relationship. In your case, $R^{2} = .089$, means that $8.9\%$ of the variation in your response variable can be explained a linear relationship with the predictor. What constitutes a "large" $R^2$ is discipline dependent. For example, in social sciences $R^2 = .2$ might be "large" but in controlled environments like a factory setting, $R^2 > .9$ may be required to say there is a "strong" relationship. In most situations $.089$ is a very small $R^2$, so your conclusion that there is a weak linear relationship is probably reasonable.

share|improve this answer
    
Thanks Macro. Very helpful answer. I'm glad you included the part about what, exactly, the p-value is testing. It makes a lot of sense that the p-value would be so low considering how close to 1 the slope is. It seems to me, in light of your answer and @jedfrancis', the r^2 value describes that 'cloud' of data points around the line of regression. Excellent! That's much more clear now! –  Mog Jul 19 '11 at 19:56
    
@Macro (+1), fine answer. But how does the "strength of the relationship" depend on the "size of the intercept"? AFAIK the intercept says nothing at all about correlation or "strength" of a linear relationship. –  whuber Jul 19 '11 at 21:32
    
@whuber, you're right - the intercept is irrelevant and definitely doesn't change the correlation - I was thinking about the regression function $y = 10000 + x$ vs. $y = x$ and thinking somehow of the second one being a stronger relationship (all else held equal), since a greater amount of the magnitude of $y$ was due to $x$ in the latter case. Doesn't make much sense now that I think about it. I've edited the post. –  Macro Jul 19 '11 at 21:36
3  
@macro Excellent answer, but I would stress (for those new to this subject) that R^2 can be very low even with a strong relationship, if the relationship is nonlinear, and particularly if it is nonmonotonic. My favorite example of this is the relationship between stress and exam score; very low stress and very high stress tend to be worse than moderate stress. –  Peter Flom Jul 20 '11 at 10:19
1  
@macro Yeah, your answer was good, but I have worked with people who don't know a lot of statistics, and I've seen what happens ... sometimes what we say is not what they hear! –  Peter Flom Jul 21 '11 at 10:42
show 3 more comments

For a linear regression, the fitted slope is going to be the correlation (which, when squared, gives the coefficient of determination, the $R^2$) times the empirical standard deviation of the regressand (the $y$) divided by the empirical standard deviation of the regressor (the $x$). Depending on the scaling of the $x$ and $y$, you can have a fit slope equal to one but an arbitrarily small $R^2$ value.

In short, the slope is not a good indicator of model 'fit' unless you are certain that the scales of the dependent and independent variables must be equal to each other.

share|improve this answer
add comment

The $R^2$ value tells you how much variation in the data is explained by the fitted model.

The low $R^2$ value in your study suggests that your data is probably spread widely around the regression line, meaning that the regression model can only explain (very little) 8.9% of the variation in the data.

Have you checked to see whether a linear model is appropriate? Have a look at the distribution of your residuals, as you can use this to assess the fit of the model to your data. Ideally, your residuals should not show a relation with your $x$ values, and if it does, you may want to think of rescaling your variables in a suitable way, or fitting a more appropriate model.

share|improve this answer
    
Thanks @jed. Yes, I'd checked the normality of the residuals, and all was well. Your suggestion that the data is spread widely around that regression line is exactly right - the data points looks like a cloud around the line of regression plotted by the software. –  Mog Jul 19 '11 at 19:51
1  
Welcome to our site, @jed, and thanks for your reply! Please note that the slope itself says almost nothing about the correlation, apart from its sign, because correlation does not depend on the units in which X and Y are measured but the slope does. –  whuber Jul 19 '11 at 21:29
1  
@whuber is saying that the value of the slope does not tell you anything about the strength of the association unless variables are standardized. See shabbychefs answer. –  wolf.rauch Jul 19 '11 at 23:54
    
@wolf.rauch gotcha –  jedfrancis Jul 20 '11 at 1:56
    
@jed It would be good if you were to correct your reply. –  whuber Jul 20 '11 at 13:08
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.