Linked Questions
16 questions linked to/from How to calculate out of sample R squared?
21
votes
4
answers
3k
views
Why does regularization wreck orthogonality of predictions and residuals in linear regression?
Following up on this question...
In ordinary least squares, the predictions and residuals are orthogonal. $$\sum_{i=1}^n\hat{y}_i (y_i - \hat{y}_i) = 0$$
If we estimate the regression coefficients ...
- 67.1k
16
votes
3
answers
39k
views
Should $ R^2$ be calculated on training data or test data?
When calculating the $R^2$ value of a linear regression model, should it be calculated on the training dataset, test dataset or both and why?
Furthermore, when calculating $SS_{\text{res}}$ and $SS_{\...
- 1,334
15
votes
2
answers
3k
views
How to motivate the definition of $R^2$ in `sklearn.metrics.r2_score`?
TLDR: What motivates the definition of $R^2$ in the Python function sklearn.metrics.r2_score?
DETAILS
The Python machine learning package ...
- 67.1k
12
votes
1
answer
3k
views
Interpreting nonlinear regression $R^2$
In ordinary least squares linear regression, $R^2=1-\frac{SSRes}{SSTotal}$ is described as the “proportion of variance explained”. Does this apply to nonlinear regression, too?
- 67.1k
2
votes
2
answers
4k
views
High AUC but low R squared in a random forest classifier
I have been looking for an answer on this website and on Google but I can't seem to find a clear explanation anywhere.
The problem is the following. I built a Random Forest model (using Python's ...
- 158
2
votes
4
answers
2k
views
Is there a relation between the p-values of coefficients and the $R^2$ in an OLS regression? [closed]
I have a very simple question. I know that the R-squared is the proportion of the variance for a dependent variable that's explained by an independent variable or variables in a regression model.
I ...
- 528
2
votes
1
answer
1k
views
What is a good error function for regression on highly unbalanced data
I'm working with a dataset of bank loans, trying to predict which loans are going to default based on some pre-loan-subscription features (for instance, what's the credit grade of the borrower, or the ...
- 571
2
votes
1
answer
848
views
How to distinguish two versions of R-squared calculated on test set?
I've come across two ways that people calculate R-squared on a test set:
Calculate the square of the correlation between predictions and actual values (in practice, I've seen people do this in R by ...
- 4,404
5
votes
1
answer
517
views
Calculating pseudo-$R^2$ for out-of-sample probit model forecasts
I'm trying to replicate parts of:
Estrella, A., & Mishkin, F. S. (1998). Predicting U.S. Recessions: Financial Variables as Leading Indicators. Review of Economics and Statistics, 80(1), 45–61.
...
- 163
4
votes
1
answer
367
views
Regression hypothesis testing via out-of-sample testing
Let's consider two linear models.
$$\text{Full model}\\\mathbb{E}\big{[}Y\big{\vert} X_1,\dots,X_p, X_{p+1},\dots,X_{p+k}\big{]}=\\\beta_0 + \bigg[\beta_1X_1+\dots + \beta_pX_p\bigg] + \bigg[\beta_{p+...
- 67.1k
1
vote
1
answer
438
views
Adjusted R2 for LSTM
Background: I am working on a problem, where I am making predictions for a time-series data. I am considering two approaches:
Use LSTM, predict n samples using recursive strategy (suggested e.g. in ...
- 11
1
vote
1
answer
218
views
Does it make sense to calculate R2 on splits of test data based on target value percentile?
I have an XGBoost regression model to predict a numeric target y. y is quite right-skewed when I plot histogram. For example, ...
- 131
0
votes
1
answer
103
views
3
votes
1
answer
118
views
Out-of-sample R square is NEGATIVE [closed]
The "Out-of-sample $R^2$" is defined as:
$$
R^2_{OOS} = 1 - \frac{\sum_{t=\tau}^T\left(Y_t - \hat{Y}_{t\vert t-1}\right)^2}{\sum_{t=\tau}^T\left(Y_t - \hat{\mu}_{t\vert t-1}\right)^2}
$$
...
- 31
2
votes
1
answer
98
views
Which Forecast Evaluation Metric To Use?
It is a forecasting problem. I need an evaluation metric which penalizes under-predictions more than over-predictions. Also I want it's range in certain interval (say 0-100), so that it becomes easier ...
