Return to Answer

I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of determination $R^2$ as is standard for reporting regression models and as can be calculated for any model that yields predicted values on the same scale as the original data.

That is, $R^2$ is essentially the square of the correlation between observed and predicted outcomes, although statistical people vary in how much emphasis they put on that. (There is much small print that others might add there, and they're welcome to do that.) A side comment is that here lies small scope for mischief over what is reported. So to naive readers $R^2$ of say 0.04 might look like disaster, while a correlation of 0.2 might seem less disastrous, but the results are naturally one and the same.

Watch out for many popular analogues of or alternatives to $R^2$ that, although sometimes helpful, are not squared correlations. (Labels like pseudo- may be visible.)

how to calculate R-squared in glm? is one of several threads that serves to signal one possibility as well as a range of views in this territory.

That said, there are many more reservations here.

1. Technically, if your fitting procedure does not maximize $R^2$ as an explicit goal or as a side-effect of some other criterion (e.g. maximum likelihood), then there has to be a flag that you're evaluating a model by a criterion not used in fitting, which is going to be somewhere between acceptable or pragmatic and dubious or irrelevant. (Imagine evaluating sports people by how much they smile or their hair style rather than how well they did. Such irrelevance may be entertaining at best, but it is a side-issue.)

2. A common objection to $R^2$, or at least a standard comment, is that it needs substantive interpretation. (Naturally all figures of merit need such interpretation.) Thus an $R^2$ that isn't very close to 1 in some fields signals experimental incompetence. In others an $R^2$ very close to 1 may signal outrageous over-fitting, fraud, or a very silly question. Further, in many fields $R^2$ being close to 0 doesn't rule out a model being interesting or useful. So simple predictors such as age or gender don't usually get you far in predicting academic performance, but it can still be of concern whether age or gender has a discernible effect. This last situation seems common in social science or medical applications.

3. Correlation doesn't measure agreement, just as predicting temperature by the temperature multiplied by a positive constant gives you a perfect correlation, but that is not a helpful prediction. In short, bias can also be a problem.

In general, and more positively, I often want to couple a scaled measure like $R^2$ with a scaled measure like MSE (or greatly preferable RMSE) that is on the same scale as the original outcome. It is quite common that mischievous researchers cite a measure that makes their model look good and fail to cite any other. If people's heights are predicted to an RMSE of 0.1 mm or of 10 cm, then in either case I know that a model is useless, although for different reasons. Researchers should have a good feeling for the units of measurement used in their field.

I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of determination $R^2$, as is standard for reporting regression models and as can be calculated for any model that yields predicted values on the same scale as the original data.

That is, $R^2$ is essentially the square of the correlation between observed and predicted outcomes, although statistical people vary in how much emphasis they put on that. (There is much small print that others might add there, and they're welcome to do that.) A side comment is that here lies small scope for mischief over what is reported. So to naive readers $R^2$ of say 0.04 might look like disaster, while a correlation of 0.2 might seem less disastrous, but the results are naturally one and the same.

Watch out for many popular analogues of or alternatives to $R^2$ that, although sometimes helpful, are not squared correlations. (Labels like pseudo- may be visible.)

how to calculate R-squared in glm? is one of several threads that serves to signal one possibility as well as a range of views in this territory.

That said, there are many more reservations here.

1. Technically, if your fitting procedure does not maximize $R^2$ as an explicit goal or as a side-effect of some other criterion (e.g. maximum likelihood), then there has to be a flag that you're evaluating a model by a criterion not used in fitting, which is going to be somewhere between acceptable or pragmatic and dubious or irrelevant. (Imagine evaluating sports people by how much they smile or their hair style rather than how well they did. Such irrelevance may be entertaining at best, but it is a side-issue.)

2. A common objection to $R^2$, or at least a standard comment, is that it needs substantive interpretation. (Naturally all figures of merit need such interpretation.) Thus an $R^2$ that isn't very close to 1 in some fields signals experimental incompetence. In others an $R^2$ very close to 1 may signal outrageous over-fitting, fraud, or a very silly question. Further, in many fields $R^2$ being close to 0 doesn't rule out a model being interesting or useful. So simple predictors such as age or gender don't usually get you far in predicting academic performance, but it can still be of concern whether age or gender has a discernible effect. This last situation seems common in social science or medical applications.

3. Correlation doesn't measure agreement, just as predicting temperature by the temperature multiplied by a positive constant gives you a perfect correlation, but that is not a helpful prediction. In short, bias can also be a problem.

4. No single measure of model performance can capture all that is important about a model's virtues and limitations.

In general, and more positively, I often want to couple a scaled measure like $R^2$ with a scaled measure like mean square error MSE (or, greatly preferable, its root RMSE). It can be very helpful to get measures on the same scale as the original outcome. It is quite common that mischievous researchers cite a measure that makes their model look good and fail to cite any other. If people's heights are predicted to an RMSE of 0.1 mm or of 10 cm, then in either case I know that a model is useless, although for different reasons. Researchers should have a good feeling for the units of measurement used in their field.

I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of determination $R^2$ as is standard for reporting regression models and as can be calculated for any model that yields predicted values on the same scale as the original data. Watch out, however, for many analogues of $R^2$ that, although sometimes helpful, are not squared correlations. (Labels like pseudo- may be visible.)

how to calculate R-squared in glm? is one of several threads that serves to signal one possibility as well as a range of views in this territory.

That said, there are many more reservations here.

1. Technically, if your fitting procedure does not maximize $R^2$ as an explicit goal or as a side-effect of some other criterion (e.g. maximum likelihood), then there has to be a flag that you're evaluating a model by a criterion not used in fitting, which is going to be somewhere between acceptable or pragmatic and dubious or irrelevant. (Imagine evaluating sports people by how much they smile or their hair style rather than how well they did. Such irrelevance may be entertaining at best, but it is a side-issue.)

2. A common objection to $R^2$, or at least a standard comment, is that it needs substantive interpretation. (Naturally all figures of merit need such interpretation.) Thus an $R^2$ that isn't very close to 1 in some fields signals experimental incompetence. In others an $R^2$ very close to 1 may signal outrageous over-fitting, fraud, or a very silly question. Further, in many fields $R^2$ being close to 0 doesn't rule out a model being interesting or useful. So simple predictors such as age or gender don't usually get you far in predicting academic performance, but it can still be of concern whether age or gender has a discernible effect. This last situation seems common in social science or medical applications.

3. Correlation doesn't measure agreement, just as predicting temperature by the temperature multiplied by a positive constant gives you a perfect correlation, but that is not a helpful prediction. In short, bias can also be a problem.

In general, and more positively, I often want to couple a scaled measure like $R^2$ with a scaled measure like MSE (or greatly preferable RMSE) that is on the same scale as the original outcome. It is quite common that mischievous researchers cite a measure that makes their model look good and fail to cite any other. If people's heights are predicted to an RMSE of 0.1 mm or of 10 cm, then in either case I know that a model is useless, although for different reasons. Researchers should have a good feeling for the units of measurement used in their field.

I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of determination $R^2$ as is standard for reporting regression models and as can be calculated for any model that yields predicted values on the same scale as the original data.

That is, $R^2$ is essentially the square of the correlation between observed and predicted outcomes, although statistical people vary in how much emphasis they put on that. (There is much small print that others might add there, and they're welcome to do that.) A side comment is that here lies small scope for mischief over what is reported. So to naive readers $R^2$ of say 0.04 might look like disaster, while a correlation of 0.2 might seem less disastrous, but the results are naturally one and the same.

Watch out for many popular analogues of or alternatives to $R^2$ that, although sometimes helpful, are not squared correlations. (Labels like pseudo- may be visible.)

how to calculate R-squared in glm? is one of several threads that serves to signal one possibility as well as a range of views in this territory.

That said, there are many more reservations here.

1. Technically, if your fitting procedure does not maximize $R^2$ as an explicit goal or as a side-effect of some other criterion (e.g. maximum likelihood), then there has to be a flag that you're evaluating a model by a criterion not used in fitting, which is going to be somewhere between acceptable or pragmatic and dubious or irrelevant. (Imagine evaluating sports people by how much they smile or their hair style rather than how well they did. Such irrelevance may be entertaining at best, but it is a side-issue.)

2. A common objection to $R^2$, or at least a standard comment, is that it needs substantive interpretation. (Naturally all figures of merit need such interpretation.) Thus an $R^2$ that isn't very close to 1 in some fields signals experimental incompetence. In others an $R^2$ very close to 1 may signal outrageous over-fitting, fraud, or a very silly question. Further, in many fields $R^2$ being close to 0 doesn't rule out a model being interesting or useful. So simple predictors such as age or gender don't usually get you far in predicting academic performance, but it can still be of concern whether age or gender has a discernible effect. This last situation seems common in social science or medical applications.

3. Correlation doesn't measure agreement, just as predicting temperature by the temperature multiplied by a positive constant gives you a perfect correlation, but that is not a helpful prediction. In short, bias can also be a problem.

In general, and more positively, I often want to couple a scaled measure like $R^2$ with a scaled measure like MSE (or greatly preferable RMSE) that is on the same scale as the original outcome. It is quite common that mischievous researchers cite a measure that makes their model look good and fail to cite any other. If people's heights are predicted to an RMSE of 0.1 mm or of 10 cm, then in either case I know that a model is useless, although for different reasons. Researchers should have a good feeling for the units of measurement used in their field.

I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of determination $R^2$ as is standard for reporting regression models and as can be calculated for any model that yields predicted values on the same scale as the original data. Watch out, however, for many analogues of $R^2$ that, although sometimes helpful, are not squared correlations. (Labels like pseudo- may be visible.)

how to calculate R-squared in glm? is one of several threads that serves to signal one possibility as well as a range of views in this territory.

That said, there are many more reservations here.

1. Technically, if your fitting procedure does not maximize $R^2$ as an explicit goal or as a side-effect of some other criterion (e.g. maximum likelihood), then there has to be a flag that you're evaluating a model by a criterion not used in fitting, which is going to be somewhere between acceptable or pragmatic and dubious or irrelevant. (Imagine evaluating sports people by how much they smile or their hair style rather than how well they did. Such irrelevance may be entertaining at best, but it is a side-issue.)

2. A common objection to $R^2$, or at least a standard comment, is that it needs substantive interpretation. (Naturally all figures of merit need such interpretation.) Thus an $R^2$ that isn't very close to 1 in some fields signals experimental incompetence. In others an $R^2$ very close to 1 may signal outrageous over-fitting, fraud, or a very silly question. Further, in many fields $R^2$ being close to 0 doesn't rule out a model being interesting or useful. So simple predictors such as age or gender don't usually get you far in predicting academic performance, but it can still be of concern whether age or gender has a discernible effect. This last situation seems common in social science or medical applications.

3. Correlation doesn't measure agreement, as just predicting temperature by the temperature multiplied by a positive constant gives you a perfect correlation, but that is not a helpful prediction.

In general, and more positively, I often want to couple a scaled measure like $R^2$ with a scaled measure like MSE (or greatly preferable RMSE) that is on the same scale as the original outcome. It is quite common that mischievous researchers cite a measure that makes their model look good and fail to cite any other. If people's heights are predicted to an RMSE of 0.1 mm or of 10 cm, then in either case I know that a model is useless, although for different reasons. Researchers should have a good feeling for the units of measurement used in their field.

I take it that correlation here is the correlation between observed and predicted outcome or response values. If so, then using such a correlation is just a variant on using the coefficient of determination $R^2$ as is standard for reporting regression models and as can be calculated for any model that yields predicted values on the same scale as the original data. Watch out, however, for many analogues of $R^2$ that, although sometimes helpful, are not squared correlations. (Labels like pseudo- may be visible.)

how to calculate R-squared in glm? is one of several threads that serves to signal one possibility as well as a range of views in this territory.

That said, there are many more reservations here.

1. Technically, if your fitting procedure does not maximize $R^2$ as an explicit goal or as a side-effect of some other criterion (e.g. maximum likelihood), then there has to be a flag that you're evaluating a model by a criterion not used in fitting, which is going to be somewhere between acceptable or pragmatic and dubious or irrelevant. (Imagine evaluating sports people by how much they smile or their hair style rather than how well they did. Such irrelevance may be entertaining at best, but it is a side-issue.)

2. A common objection to $R^2$, or at least a standard comment, is that it needs substantive interpretation. (Naturally all figures of merit need such interpretation.) Thus an $R^2$ that isn't very close to 1 in some fields signals experimental incompetence. In others an $R^2$ very close to 1 may signal outrageous over-fitting, fraud, or a very silly question. Further, in many fields $R^2$ being close to 0 doesn't rule out a model being interesting or useful. So simple predictors such as age or gender don't usually get you far in predicting academic performance, but it can still be of concern whether age or gender has a discernible effect. This last situation seems common in social science or medical applications.

3. Correlation doesn't measure agreement, just as predicting temperature by the temperature multiplied by a positive constant gives you a perfect correlation, but that is not a helpful prediction. In short, bias can also be a problem.

In general, and more positively, I often want to couple a scaled measure like $R^2$ with a scaled measure like MSE (or greatly preferable RMSE) that is on the same scale as the original outcome. It is quite common that mischievous researchers cite a measure that makes their model look good and fail to cite any other. If people's heights are predicted to an RMSE of 0.1 mm or of 10 cm, then in either case I know that a model is useless, although for different reasons. Researchers should have a good feeling for the units of measurement used in their field.