I ran a regression with 4 variables, and all are very statistically significant, with T values $\approx 7,9,26$ and $31$ (I say $\approx$ because it seems irrelevant to include the decimals) which are very high and clearly significant. But then the $R^2$ is only .2284. Am I misinterpreting the t values here to mean something they're not? My first reaction upon seeing the t values was that the $R^2$ would be quite high, but maybe that is a high $R^2$?
$\begingroup$
$\endgroup$
3
-
2$\begingroup$ I bet your $n$ is moderately large, right? $\endgroup$– Glen_bCommented Nov 27, 2012 at 7:37
-
$\begingroup$ @Glen_b yes, around 6000. $\endgroup$– KyleCommented Nov 27, 2012 at 15:30
-
12$\begingroup$ Then large $t$-statistics being associated with small $R^2$ is entirely unremarkable. Since standard errors decrease as $1/\sqrt{n}$, $t$-ratios will increase as $\sqrt{n}$, while $R^2$ will tend to remain constant with increasing $n$. Why do you care what the $R^2$ is? Why do you care what the t-ratios are? $\endgroup$– Glen_bCommented Nov 27, 2012 at 22:25
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
4
The $t$-values and $R^2$ are used to judge very different things. The $t$-values are used to judge the accurary of your estimate of the $\beta_i$'s, but $R^2$ measures the amount of variation in your response variable explained by your covariates. Suppose you are estimating a regression model with $n$ observations,
$$ Y_i = \beta_0 + \beta_1X_{1i} + ...+ \beta_kX_{ki}+\epsilon_i $$
where $\epsilon_i\overset{i.i.d}{\sim}N(0,\sigma^2)$, $i=1,...,n$.
Large $t$-values (in absolute value) lead you to reject the null hypothesis that $\beta_i=0$. This means you can be confident that you have correctly estimated the sign of the coefficient. Also, if $|t|$>4 and you have $n>5$, then 0 is not in a 99% confidence interval for the coefficient. The $t$-value for a coefficient $\beta_i$ is the difference between the estimate $\hat{\beta_i}$ and 0 normalized by the standard error $se\{\hat{\beta_i}\}$.
$$ t=\frac{\hat{\beta_i}}{se\{\hat{\beta_i}\}} $$
which is simply the estimate divided by a measure of its variability. If you have a large enough dataset, you will always have statistically significant (large) $t$-values. This does not mean necessarily mean your covariates explain much of the variation in the response variable.
As @Stat mentioned, $R^2$ measures the amount of variation in your response variable explained by your dependent variables. For more about $R^2$, go to wikipedia. In your case, it appears you have a large enough data set to accurately estimate the $\beta_i$'s, but your covariates do a poor job of explaining and\or predicting the response values.
-
1$\begingroup$ (+1) It is clear from the very beginning that this is a well considered, informative explanation. $\endgroup$– whuber ♦Commented Nov 27, 2012 at 8:17
-
$\begingroup$ Nice answer. I find the terms "practical significance" and "statistical significance" to often be helpful in thinking about this issue. $\endgroup$ Commented Nov 30, 2012 at 20:57
-
4$\begingroup$ There is also a simple transformation between the two statistics: $R^2=\frac{t^2}{t^2+df}$ $\endgroup$– JeffCommented Dec 16, 2015 at 21:03
-
$\begingroup$ @caburke If t stats and p values are functions of the sample size, do you have to "correct" them for this? $\endgroup$ Commented Aug 30 at 17:13
$\begingroup$
$\endgroup$
To say the same thing as caburke but more simply, you are very confidant that the average response caused by your variables is not zero. But there are lots of other things that you don't have in the regression that cause the response to jump around.
answered Nov 30, 2012 at 20:15
$\begingroup$
$\endgroup$
Could it be that although your predictors are trending linearly in terms of your response variable (slope is significantly different from zero), which makes the t values significant, but the R squared is low because the errors are large, which means that the variability in your data is large and thus your regression model is not a good fit (predictions aren't as accurate)?
Just my 2 cents.
Perhaps this post can help: http://blog.minitab.com/blog/adventures-in-statistics/how-to-interpret-a-regression-model-with-low-r-squared-and-low-p-values
$\begingroup$
$\endgroup$
2
Several answers given are close but still wrong.
"The t-values are used to judge the accurary of your estimate of the βi's" is the one that concerns me the most.
The T-value is merely an indication of the likelihood of random occurrence. Large means unlikely. Small means very likely. Positive and Negative don't matter to the likelihood interpretation.
"R2 measures the amount of variation in your response variable explained by your covariates" is correct.
(I would have commented but am not allowed by this platform yet.)
-
3$\begingroup$ You seem to write about t-values as if they were p-values. $\endgroup$– whuber ♦Commented Nov 26, 2018 at 20:12
-
$\begingroup$ Functionally aren't they the same? $\endgroup$ Commented Jan 18, 2022 at 7:11
$\begingroup$
$\endgroup$
The only way to deal with a small R squared, check the following:
- Is your sample size large enough? If yes, do step 2. but if no, increase your sample size.
How many covariates did you use for your model estimation? If more than 1 as in your case, deal with the problem of multicolinearity of the covariates or simply, run the regression again and this time without the constant which is known as beta zero.
However, if the problem still persists, then do a stepwise regression and select the model with a high R squared. But which I cannot recommend to you because it brings about bias in the covariates
