63
votes
Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$
If there is a constant term in the model then $\mathbf{1_n}$ lies in the column space of $\mathbf{X}$ (as does $\bar{Y}\mathbf{1_n}$, which will come in useful later). The fitted $\mathbf{\hat{Y}}$ is ...
- 21.2k
57
votes
What does negative R-squared mean?
$R^2$ can be negative, it just means that:
The model fits your data very badly
You did not set an intercept
To the people saying that $R^2$ is between 0 and 1, this is not the case. While a negative ...
- 753
44
votes
Is a high $R^2$ ever useless?
Yes. The criteria for evaluating a statistical model depend on the specific problem at hand and aren't some mechanical function of $R^2$ or statistical significance (though they matter). The relevant ...
- 21k
44
votes
R-squared is equal to 81% means what?
As a matter of fact, this last explanation is the best one:
r-squared is the percentage of variation in 'Y' that is accounted for by its regression on 'X'
Yes, it is quite abstract. Let's try to ...
- 99.7k
39
votes
Accepted
What is the distribution of $R^2$ in linear regression under the null hypothesis? Why is its mode not at zero when $k>3$?
For the specific hypothesis (that all regressor coefficients are zero, not including the constant term, which is not examined in this test) and under normality, we know (see eg Maddala 2001, p. 155, ...
- 54.1k
32
votes
Accepted
Showing machine learning results are statistically irrelevant
You answered yourself:
I made two additional models (mean and last sample) which often match or beat the RMSE of the RF and ANN models published in the paper. The mean model just takes the mean of ...
Tim♦
- 117k
31
votes
Relationship between $R^2$ and correlation coefficient
One way of interpreting the coefficient of determination $R^{2}$ is to look at it as the Squared Pearson Correlation Coefficient between the observed values $y_{i}$ and the fitted values $\hat{y}_{i}$....
- 1,231
31
votes
Accepted
Is there any difference between $r^2$ and $R^2$?
Notation on this matter seems to vary a little.
$R$ is used in the context of multiple correlation and is called the "multiple correlation coefficient". It is the correlation between the ...
- 21.2k
29
votes
What is the mathematical relationship between R2 and MSE?
Yes, allow me to elaborate.
Recall that for some outcome $y_i \in \mathbb{R}, \forall i=1,2,..,n$ we define MSE and $\textrm{R}^2$ as
\begin{equation}
\textrm{MSE}(y, \hat{y} ) = \frac{1}{n} \sum_{i=1}...
- 602
28
votes
What does it mean for a linear regression to be statistically significant but has very low r squared?
It means that you can explain a small portion of the variance in the data. For instance, you can establish that a college degree impacts salaries, but at the same time it's just a small factor. There ...
- 56.8k
27
votes
Accepted
R-squared in quantile regression
Koenker and Machado$^{[1]}$ describe $R^1$, a local measure of goodness of fit at the particular ($\tau$) quantile.
Let $V(\tau) = \min_{b}\sum \rho_\tau(y_i-x_i'b)$
Let $\hat{\beta}(\tau)$ and $\...
- 263k
27
votes
Accepted
Adding a linear regression predictor decreases R squared
Could it be that you have missing values in Q that are getting auto-dropped? That'd have implications on the sample, making the two regressions not comparable.
- 12.3k
27
votes
Accepted
Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?
This answer will mostly focus on $R^2$, but most of this logic extends to other metrics such as AUC and so on.
This question can almost certainly not be answered well for you by readers at ...
- 13.2k
26
votes
Accepted
Difference between selecting features based on "F regression" and based on $R^2$ values?
TL:DR
There won't be a difference if F-regression just computes the F statistic and pick the best features. There might be a difference in the ranking, assuming <...
- 3,461
24
votes
Is R-squared value appropriate for comparing models?
I think the crucial part to consider in answering your question is
I'm trying to identify the best model to predict the prices of automobiles
because this statement implies something about why you ...
- 1,491
23
votes
R-squared in quantile regression
The pseudo-$R^2$ measure suggested by Koenker and Machado (1999) in JASA measures goodness of fit by comparing the sum of weighted deviations for the model of interest
with the same sum from a model ...
- 31.6k
23
votes
What is the distribution of $R^2$ in linear regression under the null hypothesis? Why is its mode not at zero when $k>3$?
I won't rederive the $\mathrm{Beta}(\frac{k-1}{2}, \, \frac{n-k}{2})$ distribution in @Alecos's excellent answer (it's a standard result, see here for another nice discussion) but I want to fill in ...
- 21.2k
23
votes
What does negative R-squared mean?
Neither answer so far is entirely correct, so I will try to give my understanding of R-Squared. I have given a more detailed explanation of this on my blog post here "What is R-Squared"
Sum Squared ...
- 997
23
votes
Why is $SST=SSE + SSR$? (One variable linear regression)
Adding and subtracting gives
\begin{eqnarray*}
\sum_{i=1}^n (y_i-\bar y)^2&=&\sum_{i=1}^n (y_i-\hat y_i+\hat y_i-\bar y)^2\\
&=&\sum_{i=1}^n (y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)...
- 27.1k
23
votes
Accepted
Why is R Squared not a good measure for regressions fit using LASSO?
The goal of using LASSO is obtaining a sparse representation (of a predicted quantity) in the sense of not having many covariates. Comparing models with $R^2$ tends to favor models with lots of ...
- 53.6k
21
votes
Accepted
What is the problem with using R-squared in time series models?
Some aspects of the issue:
If somebody gives us a vector of numbers $\mathbf y$ and a conformable matrix of numbers $\mathbf X$, we do not need to know what is the relation between them to execute ...
- 54.1k
21
votes
Accepted
Ridge regression in R with p values and goodness of fit
Although I recommended lm.ridge to you in response to an earlier question, you might consider the glmnet package as a better way ...
- 66.6k
21
votes
Difference between selecting features based on "F regression" and based on $R^2$ values?
I spent some time looking through the Scikit source code in order to understand what f_regression does, and I would like to post my observations here.
The original ...
- 526
21
votes
Accepted
Can standardized $\beta$ coefficients in linear regression be used to estimate the $R^2$?
The geometric interpretation of ordinary least squares regression provides the requisite insight.
Most of what we need to know can be seen in the case of two regressors $x_1$ and $x_2$ with response $...
- 294k
20
votes
Accepted
Why does statsmodels.api.OLS over-report the r-squared value?
This is not technically an error in statsmodels, rather it is because statsmodels.OLS does not add the intercept/constant term ...
- 11.7k
19
votes
R square in mixed model with random effects
The R package MuMIn also now has a function for calculating Nakagawa and Schielzeth's r-squared for mixed models. That is the function r.squaredGLMM() and you ...
- 406
18
votes
Accepted
Regressions. Why a and b explains more than a+b?
The reason is flexibility.
Option 1: When you regress $X_1, X_2$ on $Y$ you are allowing the coefficients to be different. In other words, your regression equation is $Y = \alpha + \beta_1X_1 + \...
- 7,514
18
votes
17
votes
Accepted
Should partial $R^2$ add up to total $R^2$ in multiple regression?
No.
One way to understand partial $R^2$ for a given predictor is that it equals the $R^2$ that you would get if you first regress your independent variable on all other predictors, take the residuals, ...
- 95.8k
17
votes
Accepted
Linear regression what does the F statistic, R squared and residual standard error tell us?
The best way to understand these terms is to do a regression calculation by hand. I wrote two closely related answers (here and here), however they may not fully help you understanding your particular ...
- 5,292
Only top scored, non community-wiki answers of a minimum length are eligible