28
$\begingroup$

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this pseudo R-squared?

Is it a relative comparison for nested models (e.g. a 6 variable model has a McFadden's pseudo R-squared of 0.192, whereas a 5 variable model (after removing one variable from the aforementioned 6 variable model), this 5 variable model has a pseudo R-squared of 0.131. Would we would want to keep that 6th variable in the model?) or is it an absolute quantity (e.g. a given model that has a McFadden's pseudo R-squared of 0.192 is better than any existing model with a McFadden's pseudo R-squared of 0.180 (for even non-nested models)? These are just possible ways to look at McFadden’s pseudo R-squared; however, I assume these two views are way off, thus the reason why I am asking this question here.

I have done a great deal of research on this topic, and I have yet to find the answer that I am looking for in terms of being able to interpret a McFadden's pseudo R-squared of 0.192. Any insight and/or references are greatly appreciated! Before answering this question, I am aware that this isn't the best measure to describe a logistic regression model, but I would like to have a greater understanding of this statistic regardless!

$\endgroup$
33
$\begingroup$

So I figured I'd sum up what I've learned about McFadden's pseudo R2 as a proper answer.

The seminal reference that I can see for McFadden's pseudo R2 is: McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. http://eml.berkeley.edu/~mcfadden/travel.html Figure 5.5 shows the relationship between rho-squared and traditional R2 measures from OLS. My interpretation is that larger values of rho-squared (McFadden's pseudo R2) are better than smaller ones.

The interpretation of McFadden's pseudo R2 between 0.2-0.4 comes from a book chapter he contributed to: Bahvioural Travel Modelling. Edited by David Hensher and Peter Stopher. 1979. McFadden contributed Ch. 15 "Quantitative Methods for Analyzing Travel Behaviour on Individuals: Some Recent Developments". Discussion of model evaluation (in the context of multinomial logit models) begins on page 306 where he introduces rho-squared (McFadden's pseudo R2). McFadden states "while the R2 index is a more familiar concept to planner who are experienced in OLS, it is not as well behaved as the rho-squared measure, for ML estimation. Those unfamiliar with rho-squared should be forewarned that its values tend to be considerably lower than those of the R2 index...For example, values of 0.2 to 0.4 for rho-squared represent EXCELLENT fit."

So basically, rho-squared can be interpreted like R2, but don't expect it to be as big. And values from 0.2-0.4 indicate (in McFadden's words) excellent model fit.

$\endgroup$
  • $\begingroup$ Good wrap-up, Chris. Thank you for your efforts! $\endgroup$ – Matt Reichenbach May 21 '14 at 21:21
13
$\begingroup$

McFadden's R squared is defined as 1-l_mod/l_null, where l_mod is the log likelihood value for the fitted model and l_null is the log likelihood for the null model which includes only an intercept as predictor (so that every individual is predicted the same probability of 'success').

For a logistic regression model the log likelihood value is always negative (because the likelihood contribution from each observation is a probability between 0 and 1). If your model doesn't really predict the outcome better than the null model, l_mod will not be much larger than l_null, and so l_mod/l_null is approximately 1, and McFadden's R squared is close to 0 (your model has no predictive value).

Conversely if your model was really good, those individuals with a success (1) outcome would have a fitted probability close to 1, and vice versa for those with a failure (0) outcome. In this case if you go through the likelihood calculation the likelihood contribution from each individual for your model will be close to zero, such that l_mod is close to zero, and McFadden's R squared is close to 1, indicating very good predictive ability.

As to what can be considered a good value, my personal view is that like that similar questions in statistics (e.g. what constitutes a large correlation?), is that can never be a definitive answer. Last year I wrote a blog post about McFadden's R squared in logistic regression, which has some further simulation illustrations.

$\endgroup$
5
$\begingroup$

I did some more focused research on this topic, and I found that interpretations of McFadden's pseudo R-squared (also known as likelihood-ratio index) are not clear; however, it can range from 0 to 1, but will never reach or exceed 1 as a result of its calculation.

A rule of thumb that I found to be quite helpful is that a McFadden's pseudo R-squared ranging from 0.2 to 0.4 indicates very good model fit. As such, the model mentioned above with a McFadden's pseudo R-squared of 0.192 is likely not a terrible model, at least by this metric, but it isn't particularly strong either.

It is also important to note that McFadden's pseudo R-squared is best used to compare different specifications of the same model (i.e. nested models). In reference to the aforementioned example, the 6 variable model (McFadden’s pseudo R-squared = 0.192) fits the data better than the 5 variable model (McFadden’s pseudo R-squared = 0.131), which I formally tested using a log-likelihood ratio test, which indicates there is a significant difference (p < 0.001) between the two models, and thus the 6 variable model is preferred for the given dataset.

$\endgroup$
  • 1
    $\begingroup$ What is the reference you found which claims McFadden's R2 between 0.2 - 0.4 is a "very good" fit? $\endgroup$ – Chris May 1 '14 at 19:50
  • $\begingroup$ Btw...here is a reference and link to the original McFadden article where he defines his pseudo-R2 measure. McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. elsa.berkeley.edu/reprints/mcfadden/zarembka.pdf‎ $\endgroup$ – Chris May 1 '14 at 19:52
  • 1
    $\begingroup$ Thanks for the references. It appears that a lot of McFadden's work can be found on his Berkeley website. Below is a link to the entire book you cite above: elsa.berkeley.edu/users/mcfadden/travel.html All chapters appear as PDF. Rho-square (McFadden's pseudo R2) is mentioned in Chapter 5. Pages 122 onwards (see equation 5.33 and the graph which follows immediately thereafter). I don't see any mention of 0.2-0.4 = "VG model fit". I will keep searching for the seminal appearance of this "rule of thumb". Thanks for your help! $\endgroup$ – Chris May 5 '14 at 12:31
  • 1
    $\begingroup$ No problem! I appreciate your curiosity and thoroughness. The exact phrase can be found at lifesciencesite.com/lsj/life1002/…, where the authors state "A goodness-of-fit using McFadden‟s pseudo r-square (ρ2) is used for fitting the overall model. McFadden suggested ρ2 values of between 0.2 and 0.4 should be taken to represent a very good fit of the model (Louviere et al., 2000)." $\endgroup$ – Matt Reichenbach May 5 '14 at 13:12
  • 4
    $\begingroup$ My institution has an electronic copy of Louviere et al (2000). "Stated Choice Methods: Analysis and Applications". Cambridge University Press. This is the reference that Lee (Life Science Journal) cites for rho-squared in {0.2-0.4} = "VG fit". On page 55 of Louviere (associated with equation 3.32) we see the following quote: "Values of rho-squared between 0.2-0.4 are considered to be indicative of extremely good model fits. Simulations by Domenich and McFadden (1975) equivalence this range to 0.7 to 0.9 for a linear function". $\endgroup$ – Chris May 5 '14 at 14:19
4
$\begingroup$

In case anyone is still interested in finding McFadden's own word, here is the link. In a footnote, McFadden (1977, p.35) wrote that "values of .2 to .4 for [$\rho^2$] represent an excellent fit." The paper is available online.

http://cowles.yale.edu/sites/default/files/files/pub/d04/d0474.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.