5
$\begingroup$

I am wondering about the relation between the explained variance and the regression coefficients in logistic regression. So given a multiple linear regression

$y_i = \beta_0 + \sum_{k=1}^{K} \beta_k x_{ki} + e_i$,

I know that $R^2$ can be simplified to

$ R^2 = \sum_{k=1}^{K} \beta^2_k + 2 \sum_{k<k'}^{} \beta_k \beta_{k'} \rho_{kk'}$.

So given that we now the values of the regression coefficients and their interrelation $\rho_{kk'}$ we can determine $R^2$ for the linear case. Is there an equivalent in logistic regression? How would it work?

$\endgroup$
3
  • $\begingroup$ Since in logistic regression, most R2 meaures are a function of the full model's likelihood and an intercept model's likelihood, I guess it is about the relation between the regression coefficients and these two likelihood functions... $\endgroup$ Mar 15, 2018 at 14:14
  • $\begingroup$ Do you have a reference for that formula for R-squared? $\endgroup$ Mar 30, 2019 at 12:15
  • 1
    $\begingroup$ That formula for $R^2$ cannot be correct. It depends on the scale of the variables, and it’s not even bounded above. $\endgroup$ Mar 30, 2019 at 14:14

1 Answer 1

4
$\begingroup$

The best basis for explained variation in binary Y is the variance of predicted probabilities. This and related measures are discussed here which references important articles by Kent & O'Quigley and Choodari-Oskooei et al.

It doesn't help very much to expand the formula as you did, but the analogy to ordinary linear models is very helpful. Think of partitioning sum of squares total into sum of squares regression and sum of squared errors: SST = SSR + SSE. $R^2$ is essentially var(predictions) / var(raw Y). var(predictions) is easy, and for non-ordinary linear models we have to work on var(raw Y) as Kent & O'Quigley did. The blog article goes more into this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.