1
$\begingroup$

I am conducting logistic regression and got 0.1542 as pseudo R2, and it is based on Mcfadden's.

I've searched materials about this, because this is my first time that modelling logit models, and found that it has been mentioned 0.2~0.4 means 'excellent fit'.

Then can I think that it is very difficult getting high value of pseudo R2 compare to that of R2?(for example, ofc it depends on the field, but as we generally we talk about 0.6 has great fit...)

And if it's true, isn't there any matter if I think this is acceptable enough, although it is not that excellent? Or is there any table or materials that tells which pseudo R2 correspond to which R2?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Does stats.stackexchange.com/questions/3559 answer your questions? $\endgroup$
    – whuber ♦
    Dec 17, 2022 at 16:57
  • 1
    $\begingroup$ Although I have to review this page thoroughly as soon as possible, but thank you. I found some implications after I glanced at it. $\endgroup$
    – hogu
    Dec 17, 2022 at 18:04
  • $\begingroup$ It's worth noting that Ben-Akiva and Watanatada (1981) first proposed a pseudo-rsquared metric following McFadden's GEV model (1973), as discussed in Small, A Discrete Choice Model for Ordered Alternatives (Econometrica, 55, 2, 1987). Harrell has pointed out the distinction between goodness-of-fit metrics based on calibration data versus more 'independent' metrics based on out-of-sample (test or holdout) data. While both metric types are useful, the distinction is important. GOF metrics based on calibration data are subject to optimistic bias, inflating the results. $\endgroup$
    – Mike Hunter
    Dec 17, 2022 at 18:16

1 Answer 1

Reset to default
1
$\begingroup$

Even in linear regression, such an $R^2$ value can be totally fine, so I would not immediately dismiss your logistic regression model due to poor performance.

I kind of like $R^2$-style measures, but a big drawback that I see to them is viewing them like grades in school, where $0.5$ is an $F$-grade that makes us sad while $0.9$ is an $A$-grade that makes us happy, even through, for some problems, $0.4$ is outstanding performance, while $0.9$ could be rather pedestrian performance for other problems.

Your performance is what it is. It is worse than a model that achieves $0.3$ and better than a model that achieves $0.1$. If it does something useful for you, that seems like a win. If your model is not useful, perhaps because competing models have achieved superior $R^2_{McFadden}$, then your performance is not good enough.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Yes, I was also thinking like that but with your comment got confidence. But anyway, by rule of thumb, is it harder that getting good value of pseudo R2 than R2? And in other materials also tell that interpreting pseudo R2 is a bit harder so some people doesn't recommend it. So is it ok if I think 0.1542 pseudo R2 corresponds to R2 of 0.4~0.6? $\endgroup$
    – hogu
    Dec 17, 2022 at 17:51
  • $\begingroup$ @hogu McFadden himself said that $0.2$ to $0.4$ should typically be considered an excellent fit. Do you consider those excellent fits for a model evaluated with the usual $R^2?$ // If you want an $R^2$-looking metric for your model, calculate the usual $R^2$. It doesn’t correspond to the log-likelihood, but it corresponds to the Brier score, which is a reasonable metric and even a strictly proper scoring rule. $\endgroup$
    – Dave
    Dec 17, 2022 at 17:54
  • $\begingroup$ I agree with that to a certain degree, although my personal criteria is a bit higher(0.3~). So I guess you are talking that regardless all of them, just presenting that value is ok and I don't have to how much it is if it is converted to R2. $\endgroup$
    – hogu
    Dec 17, 2022 at 18:00
  • $\begingroup$ McFadden is a different measure than the usual $R^2$. Why do you see a need to compare your McFadden value to the usual $R^2?$ @hogu $\endgroup$
    – Dave
    Dec 17, 2022 at 18:11
  • $\begingroup$ Because I only knew that '0.2~0.4 is very good fit', wanted to know then whether it is also very good if pseudo R2 is about 0.15. And yes, since the measures are different I thought that I must not judge according to generally accepted 'good R2'. The reason I am mentioning R2 is because just it is familiar with me so that I wanted some figurative explanation. But I decided not to obsessed to it anymore. $\endgroup$
    – hogu
    Dec 17, 2022 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.