Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

tl;dr - for OLS regression, does a higher R-squared also imply a higher P-value? Specifically for a single explanatory variable (Y = a + bX + e) but would also be interested to know for n multiple explanatory variables (Y = a + b1X + ... bnX + e).

Context - I'm performing OLS regression on a range of variables and am trying to develop the best explanatory functional form by producing a table containing the R-squared values between the linear, logarithmic, etc., transformations of each explanatory (independent) variable and the response (dependent) variable. This looks a bit like:

Variable name --linear form-- --ln(variable) --exp(variable)-- ...etc

Variable 1 ------- R-squared ----R-squared ----R-squared --
...etc...

I'm wondering if R-squared is appropriate or if P-values would be better. Presumably there is some relationship, as a more significant relationship would imply higher explanatory power, but not sure if that is true in a rigorous way.

share|improve this question
4  
Also of interest: Is R^2 useful or dangerous?. –  whuber Feb 20 '13 at 18:43

2 Answers 2

The answer is no, there is no such regular relationship between $R^2$ and the overall regression p-value, because $R^2$ depends as much on the variance of the independent variables as it does on the variance of the residuals (to which it is inversely proportional), and you are free to change the variance of the independent variables by arbitrary amounts.

As an example, consider any set of multivariate data $((x_{i1}, x_{i2}, \ldots, x_{ip}, y_i))$ with $i$ indexing the cases and suppose that the set of values of the first independent variable, $\{x_{i1}\}$, has a unique maximum $x^*$ separated from the second-highest value by a positive amount $\epsilon$. Apply a non-linear transformation of the first variable that sends all values less than $x^* - \epsilon/2$ to the range $[0,1]$ and sends $x^*$ itself to some large value $M \gg 1$. For any such $M$ this can be done by a suitable (scaled) Box-Cox transformation $x \to a((x-x_0)^\lambda - 1)/(\lambda-1))$, for instance, so we're not talking about anything strange or "pathological." Then, as $M$ grows arbitrarily large, $R^2$ approaches $1$ as closely as you please, regardless of how bad the fit is, because the variance of the residuals will be bounded while the variance of the first independent variable is asymptotically proportional to $M^2$.


You should instead be using goodness of fit tests (among other techniques) to select an appropriate model in your exploration: you ought to be concerned about the linearity of the fit and of the homoscedasticity of the residuals. And don't take any p-values from the resulting regression on trust: they will end up being almost meaningless after you have gone through this exercise, because their interpretation assumes the choice of expressing the independent variables did not depend on the values of the dependent variable at all, which is very much not the case here.

share|improve this answer

This answer doesn't directly deal with the central question; it's nothing more than some additional information that's too long for a comment.

I point this out because econometricstatsquestion will no doubt encounter this information, or something like it at some point (stating that $F$ and $R^2$ are related) and wonder if the information given in other answers here is wrong - it's not wrong - but I think it pays to be clear about what's going on.

There is a relationship under a particular set of circumstances; if you hold the number of observations and the number of predictors fixed for a given model, $F$ is in fact monotonic in $R^2$, since

$$ F = \frac{R^2/(k-1)}{(1-R^2)/(N-k)} $$

(If you divide numerator and denominator by $R^2$, and pull the constants in $k$ out, you can see that $1/F \propto 1/R^2 - 1$ if you hold $N$ and $k$ constant.)

Since for fixed d.f. $F$ and the p-value are monotonically related, $R^2$ and the $p$-value are also monotonically related.

But change almost anything about the model, and that relationship doesn't hold across the changed circumstances.

For example, adding a point makes $(N-k)/(k-1)$ larger and removing one makes it smaller but doing either can increase or decrease $R^2$, so it looks like $F$ and $R^2$ don't necessarily move together if you add or delete data. Adding a variable decreases $(N-k)/(k-1)$ but increases $R^2$ (and vice-versa), so again, $R^2$ is not necessarily related to $F$ when you do that.

Clearly, once you compare $R^2$ and $p$-values across models with different characteristics, this relationship doesn't necessarily hold, as whuber proved in the case of nonlinear transformations.

share|improve this answer
    
I don't disagree with you, but it looks like you're answering a different question than I did. It took some reading, but I concluded that the question asks about what relationship, if any, holds between $p$ and $R^2$ when (caeteris paribus) independent variables are nonlinearly transformed. It is only when those variables are left unchanged--or, at most, linearly transformed among themselves--that we can say anything at all about such a relationship. That is part of the sense in which I think your qualifier "for a given model" has to be understood. –  whuber Feb 21 '13 at 8:47
    
I am answering a different question; and I believe your interpretation of the meaning is correct. I was more worried that such an issue as the one I raised would lead to confusion if not explained. All your points hold, to my understanding. (Now I'm concerned, in fact, that perhaps my answer doesn't serve to clarify, as I had hoped, but merely confuses the issue. Do you think there's a suitable modification that would help it? Should I delete it?) –  Glen_b Feb 21 '13 at 14:05
    
I would hate to see it deleted, Glen. If you intend to make changes, consider more explicitly pointing out which aspects of this issue you are writing about (e.g., what precisely you mean by a "given model" and what you have in mind about models with "different characteristics"). This was the spirit (collaborative, not critical) in which I offered my comment. –  whuber Feb 21 '13 at 15:13
    
I didn't feel criticized by you - you seemed to be clarifying and nothing more - but the need for it highlights an inadequacy in the answer I had been concerned about before you commented. The vagueness of 'different characteristics' are because it's a pretty general thing - vary much of anything (I even give examples of something as simple as removing a point or adding a variable to illustrate how little one needs to change) can make that monotonic relationship evaporate. I'll think about what more I might say. –  Glen_b Feb 21 '13 at 16:15
    
+1 for the edit: these are valuable comments and it's especially useful to see the formula for $F$ appear. –  whuber Feb 21 '13 at 22:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.