What is a correct way to compare two $R^2$? I have dependent variable $Y$ and $X_1, X_2, X_3, X_4.$ I run two regression models, namely with $X_1$, $X_2$ and $X_3$, $X_4$. Both $R^2$ values are close. So how I determine if they are statistically different (or the same)? How to get F-statistic for these two?
$\begingroup$
$\endgroup$
9
-
1$\begingroup$ Perhaps you could bootstrap confidence intervals for each $R^2$ and see whether they overlap, and if so, by how much? $\endgroup$– Richard HardyCommented Jan 31, 2015 at 19:46
-
$\begingroup$ Ya, that might be one of the possible solutions, but I'm really looking for a way to find F-statistic and determine p-value for them being equal. $\endgroup$– Donatas GudauskasCommented Jan 31, 2015 at 19:51
-
$\begingroup$ Another option would be to test restrictions on a nesting model. You would estimate a regression of $Y$ on all $X$'s and test restrictions of $X_1$, $X_2$ and of $X_3$, $X_4$. If you reject one and fail to reject the other that would be some indirect evidence for which of the pairs is more important. Another comment (somewhat off-topic): recall that statistical significance depends on sample size. In a large-enough sample anything will be significant (while you can almost always avoid finding statistical significance if you use a small-enough sample). $\endgroup$– Richard HardyCommented Jan 31, 2015 at 20:10
-
$\begingroup$ I have found F-statistic and p-values for restricted/unrestricted regression. But that only adds incremental information. I mean I test Y=x1+x2+x3+x4 against Y=x3+x4. And it just proves that x3 and x4 adds some explanatory information to the regression, it does not show if that information is the same. $\endgroup$– Donatas GudauskasCommented Jan 31, 2015 at 20:21
-
1$\begingroup$ I doubt that a simple F-test exists for non-nested models... Anyhow, here is one more alternative: calculate the AIC values of the two competing models and see how large the difference is. There have been some suggestions of what values signal statistical significance here (see answers by Metrics and Stat). For another reference, check out the last page of this. $\endgroup$– Richard HardyCommented Jan 31, 2015 at 20:56
|
Show 4 more comments