Questions tagged [r-squared]
The coefficient of determination, usually symbolized by $R^2$, is the proportion of the total response variance explained by a regression model. Can also be used for various pseudo R-squared proposed, for instance for logistic regression (and other models.)
247
questions
181
votes
10
answers
226k
views
When is it ok to remove the intercept in a linear regression model?
I am running linear regression models and wondering what the conditions are for removing the intercept term.
In comparing results from two different regressions where one has the intercept and the ...
287
votes
6
answers
48k
views
Is $R^2$ useful or dangerous?
I was skimming through some lecture notes by Cosma Shalizi (in particular, section 2.1.1 of the second lecture), and was reminded that you can get very low $R^2$ even when you have a completely linear ...
31
votes
3
answers
12k
views
Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?
I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
12
votes
2
answers
2k
views
How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?
TLDR: What motivates the definition of $R^2$ in the Python function sklearn.metrics.r2_score?
DETAILS
The Python machine learning package ...
150
votes
3
answers
87k
views
Removal of statistically significant intercept term increases $R^2$ in linear model
In a simple linear model with a single explanatory variable,
$\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$
I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
72
votes
9
answers
88k
views
Which pseudo-$R^2$ measure is the one to report for logistic regression (Cox & Snell or Nagelkerke)?
I have SPSS output for a logistic regression model. The output reports two measures for the model fit, Cox & Snell and ...
12
votes
1
answer
2k
views
Interpreting nonlinear regression $R^2$
In ordinary least squares linear regression, $R^2=1-\frac{SSRes}{SSTotal}$ is described as the “proportion of variance explained”. Does this apply to nonlinear regression, too?
5
votes
1
answer
11k
views
Logistic regression with poor goodness of fit (hosmer lemeshow)?
I built a model with 9 categorical predictor variables. Using SPSS, my omnibus test was significant ($\chi^2$=220.01), my -2loglikelihood was 1335.2 (Nagelkerke $R^2$ 0.231), but my Hosmer and ...
170
votes
4
answers
450k
views
When is R squared negative? [duplicate]
My understanding is that $R^2$ cannot be negative as it is the square of R. However I ran a simple linear regression in SPSS with a single independent variable and a dependent variable. My SPSS output ...
7
votes
3
answers
2k
views
$R^2$ on out-sample data set
The conventional definition of $R^2$ is:
$R^2 = 1-SSE/SST$, where SSE denotes sum of squared errors and SST is total sum of squares ($n\times variance$, n being number of sample points in train set).
...
4
votes
2
answers
282
views
Is the proportion classified correctly a reasonable analogue of $R^2$ for a classification model?
Let's do some classification and evaluate the prediction quality. The easiest metric to understand is the prediction accuracy, which can be reported as the proportion classified correctly to put the ...
60
votes
6
answers
151k
views
Relationship between $R^2$ and correlation coefficient
Let's say I have two 1-dimensional arrays, $a_1$ and $a_2$. Each contains 100 data points. $a_1$ is the actual data, and $a_2$ is the model prediction. In this case, the $R^2$ value would be:
$$
R^2 = ...
34
votes
1
answer
14k
views
Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$
I am interested in the geometric meaning of the multiple correlation $R$ and coefficient of determination $R^2$ in the regression $y_i = \beta_1 + \beta_2 x_{2,i} + \dots + \beta_k x_{k,i} + \...
46
votes
6
answers
88k
views
What does negative R-squared mean?
Let's say I have some data, and then I fit the data with a model (a non-linear regression). Then I calculate the R-squared ($R^2$).
When R-squared is negative, what does that mean? Does that mean my ...
43
votes
1
answer
33k
views
What is the difference between "coefficient of determination" and "mean squared error"?
For regression problem, I have seen people use "coefficient of determination" (a.k.a R squared) to perform model selection, e.g., finding the appropriate penalty coefficient for regularization.
...
34
votes
2
answers
16k
views
What is the distribution of $R^2$ in linear regression under the null hypothesis? Why is its mode not at zero when $k>3$?
What is the distribution of the coefficient of determination, or R squared, $R^2$, in linear univariate multiple regression under the null hypothesis $H_0:\beta=0$?
How does it depend on the number ...
16
votes
5
answers
13k
views
Is R-squared truly an invalid metric for non-linear models?
I have read that R-squared is invalid for non-linear models, because the relationship that SSR + SSE = SSTotal no longer holds. Can somebody explain why this is true?
SSR and SSE are just the ...
34
votes
4
answers
27k
views
Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients
I am wondering what the exact relationship between partial $R^2$ and coefficients in a linear model is and whether I should use only one or both to illustrate the importance and influence of factors.
...
10
votes
2
answers
2k
views
Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?
In linear regression I have come across a delightful result that if we fit the model
$$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$
then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data,
$$R^...
2
votes
1
answer
160
views
How to interpret the UCLA "adjusted count" logistic regression pseudo $R^2?$
Here, UCLA gives a number of pseudo $R^2$ values for evaluating logistic regression models. Despite the issues with doing this, the last two deal with hard classifications rather than the ...
44
votes
1
answer
22k
views
Manually calculated $R^2$ doesn't match up with randomForest() $R^2$ for testing new data
I know this is a fairly specific R question, but I may be thinking about proportion variance explained, $R^2$, incorrectly. Here goes.
I'm trying to use the ...
22
votes
3
answers
32k
views
How to calculate out of sample R squared?
I know this probably has been discussed somewhere else, but I have not been able to find an explicit answer. I am trying to use the formula $R^2 = 1 - SSR/SST$ to calculate out-of-sample $R^2$ of a ...
15
votes
2
answers
4k
views
What is an unbiased estimate of population R-square?
I am interested in getting an unbiased estimate of $R^2$ in a multiple linear regression.
On reflection, I can think of two different values that an unbiased estimate of $R^2$ might be trying to ...
30
votes
4
answers
75k
views
Pseudo R squared formula for GLMs
I found a formula for pseudo $R^2$ in the book Extending the Linear Model with R, Julian J. Faraway (p. 59).
$$1-\frac{\text{ResidualDeviance}}{\text{NullDeviance}}$$.
Is this a common formula for ...
24
votes
3
answers
42k
views
How to split r-squared between predictor variables in multiple regression?
I have just read a paper in which the authors carried out a multiple regression with two predictors. The overall r-squared value was 0.65. They provided a table which split the r-squared between the ...
21
votes
2
answers
14k
views
Calculating $R^2$ in mixed models using Nakagawa & Schielzeth's (2013) R2glmm method
I have been reading about calculating $R^2$ values in mixed models and after reading the R-sig FAQ, other posts on this forum (I would link a few but I don't have enough reputation) and several other ...
28
votes
9
answers
71k
views
Measuring accuracy of a logistic regression-based model
I have a trained logistic regression model that I am applying to a testing data set. The dependent variable is binary (boolean). For each sample in the testing data set, I apply the logistic ...
11
votes
2
answers
5k
views
Why not use the R squared to measure forecast accuracy?
Why in literature usually the common accuracy measures like MAD, MSE, RMSE, MAPE ... are used. Why not use the $R^2$ (coefficient of determination)?
I was thinking about the difference: By using the ...
11
votes
1
answer
25k
views
MAPE vs R-squared in regression models
Usually regression models are evaluated using $R^2$. I understand this metric can be misleading too at times but as far as I understand the first parameter we look at is $R^2$.
There is another ...
7
votes
2
answers
8k
views
Can you calculate $R^2$ from correlation coefficents in multiple linear regression?
In simple linear regression, $R^2$ is equivalent to the squared correlation of a dependent and an independent variable. Is this also true for multiple linear regression?
For example, I measured trait ...
23
votes
4
answers
45k
views
Linear regression what does the F statistic, R squared and residual standard error tell us?
I'm really confused about the difference in meaning regarding the context of linear regression of the following terms:
F statistic
R squared
Residual standard error
I found this webstie which gave ...
19
votes
2
answers
7k
views
Why squaring $R$ gives explained variance?
This may be a basic question, but I was wondering why an $R$ value in a regression model can simply be squared to give a figure of explained variance?
I understand that $R$ coefficient can give the ...
13
votes
2
answers
16k
views
Ridge regression in R with p values and goodness of fit [closed]
Doing ridge regression in R I have discovered
linearRidge in the ridge package - which fits a model, reports coefficients and p ...
4
votes
2
answers
6k
views
Model uncertainty (model averaging) and R-Squared ($R^2$)
Is it possible to calculate r-squared for an "average model"?
Lets say I have 4 different response variables that I want to model to a set (or subset) of 4 independent variables. I'd then like to ...
2
votes
2
answers
2k
views
What does it mean if I have a high F-stat but low $R^2$?
As far as I understand, a high F-stat leads to a high $R^2$, though the converse is not true. What does it mean if I have a high F-stat and a low $R^2$?
40
votes
1
answer
27k
views
Is there any difference between $r^2$ and $R^2$?
The correlation coefficient is usually written with a capital $R$ but sometimes not. I wonder if there really is a difference between $r^2$ and $R^2$? Can $r$ mean something else than a correlation ...
26
votes
2
answers
18k
views
R-squared in quantile regression
I am using quantile regression to find predictors of 90th percentile of my data. I am doing this in R using the quantreg package. How can I determine $r^2$ for ...
24
votes
3
answers
42k
views
What is the relationship between R-squared and p-value in a regression?
tl;dr - for OLS regression, does a higher R-squared also imply a higher P-value? Specifically for a single explanatory variable (Y = a + bX + e) but would also be interested to know for n multiple ...
19
votes
3
answers
4k
views
Principal component analysis "backwards": how much variance of the data is explained by a given linear combination of the variables?
I have carried out a principal components analysis of six variables $A$, $B$, $C$, $D$, $E$ and $F$. If I understand correctly, unrotated PC1 tells me what linear combination of these variables ...
18
votes
2
answers
22k
views
Formula for 95% confidence interval for $R^2$
I googled and searched on stats.stackexchange but I cannot find the formula to calculate a 95% confidence interval for an $R^2$ value for a linear regression. Can anyone provide it?
Even better, let'...
8
votes
3
answers
915
views
Justification for and optimality of $R^2_{adj.}$ as a model selection criterion
In a recent thread, use of adjusted $R^2$ ($R^2_{adj.}$) is mentioned in the context of model selection, e.g.
The adjustment was invented as a solution to problems caused by variable selection
...
6
votes
2
answers
1k
views
Why is R2 not reported for GLMs based on last iteration of IRLS weighted least square regression with which it is fit
Given that GLMs are generally fit using iteratively reweighted least squares (based on a Fisher scoring algorithm to maximize the max likelihood objective, which is a variant of Newton-Raphson, see ...
5
votes
4
answers
9k
views
Can the coefficient of determination $R^2$ be more than one? What is its upper bound?
It is well known that if you add additional independent variables in a linear regression, the $R^2$ of the new model is at least as large as the previous model, so you obtain a lower bound for the $R^...
4
votes
1
answer
4k
views
Coefficient of determination for binary responses
D.R. Cox and Nanny Wermuth seem to suggest that the coefficient of determination (R squared) is misleading when you have binary responses, in fact if I am understanding well, they are saying that the ...
4
votes
3
answers
10k
views
Multiple Correlation Coefficient with three or more independent variables
The formula for the multiple coefficient of correlation of two independent variables ($x_1$ and $x_2$) and an dependent variables ($y$) is this:
$$R=\sqrt{\frac{r^2_{yx_1}+r^2_{yx_2}-2r_{yx_1}r_{yx_2}...
49
votes
2
answers
64k
views
What is the adjusted R-squared formula in lm in R and how should it be interpreted?
What is the exact formula used in R lm() for the Adjusted R-squared? How can I interpret it?
Adjusted r-squared formulas
There seem to exist several formulas to ...
25
votes
2
answers
18k
views
Is a weighted $R^2$ in robust linear model meaningful for goodness of fit analysis?
I estimated a robust linear model in R with MM weights using the rlm() in the MASS package. `R`` does not provide an $R^2$ value ...
20
votes
3
answers
232k
views
What's the difference between multiple R and R squared?
In linear regression, we often get multiple R and R squared. What are the differences between them?
15
votes
3
answers
67k
views
Why is adjusted R-squared less than R-squared if adjusted R-squared predicts the model better?
As far as I understand, $R^2$ explains how well the model predicts the observation. Adjusted $R^2$ is the one that takes into account more observations (or degrees of freedom). So, Adjusted $R^2$ ...
14
votes
3
answers
22k
views
Which is better: r-squared or adjusted r-squared?
I just started to learn about the following statistical measures, r-squared and adjusted r-squared and was wondering why can't we use adjusted r-squared for every regression model considering the ...