30
$\begingroup$

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this pseudo R-squared?

Is it a relative comparison for nested models (e.g. a 6 variable model has a McFadden's pseudo R-squared of 0.192, whereas a 5 variable model (after removing one variable from the aforementioned 6 variable model), this 5 variable model has a pseudo R-squared of 0.131. Would we would want to keep that 6th variable in the model?) or is it an absolute quantity (e.g. a given model that has a McFadden's pseudo R-squared of 0.192 is better than any existing model with a McFadden's pseudo R-squared of 0.180 (for even non-nested models)? These are just possible ways to look at McFadden’s pseudo R-squared; however, I assume these two views are way off, thus the reason why I am asking this question here.

I have done a great deal of research on this topic, and I have yet to find the answer that I am looking for in terms of being able to interpret a McFadden's pseudo R-squared of 0.192. Any insight and/or references are greatly appreciated! Before answering this question, I am aware that this isn't the best measure to describe a logistic regression model, but I would like to have a greater understanding of this statistic regardless!

$\endgroup$
38
$\begingroup$

So I figured I'd sum up what I've learned about McFadden's pseudo $R^2$ as a proper answer.

The seminal reference that I can see for McFadden's pseudo $R^2$ is: McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. http://eml.berkeley.edu/~mcfadden/travel.html Figure 5.5 shows the relationship between $\rho^2$ and traditional $R^2$ measures from OLS. My interpretation is that larger values of $\rho^2$ (McFadden's pseudo $R^2$) are better than smaller ones.

The interpretation of McFadden's pseudo $R^2$ between 0.2-0.4 comes from a book chapter he contributed to: Bahvioural Travel Modelling. Edited by David Hensher and Peter Stopher. 1979. McFadden contributed Ch. 15 "Quantitative Methods for Analyzing Travel Behaviour on Individuals: Some Recent Developments". Discussion of model evaluation (in the context of multinomial logit models) begins on page 306 where he introduces $\rho^2$ (McFadden's pseudo $R^2$). McFadden states "while the $R^2$ index is a more familiar concept to planner who are experienced in OLS, it is not as well behaved as the $\rho^2$ measure, for ML estimation. Those unfamiliar with $\rho^2$ should be forewarned that its values tend to be considerably lower than those of the $R^2$ index...For example, values of 0.2 to 0.4 for $\rho^2$ represent EXCELLENT fit."

So basically, $\rho^2$ can be interpreted like $R^2$, but don't expect it to be as big. And values from 0.2-0.4 indicate (in McFadden's words) excellent model fit.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Good wrap-up, Chris. Thank you for your efforts! $\endgroup$ – Matt Reichenbach May 21 '14 at 21:21
  • $\begingroup$ I arrived late to the discussion, but I will leave this link where they explain the R2 MacFadden compared to other adjustment measures: statisticalhorizons.com/r2logistic $\endgroup$ – sergiouribe Dec 4 '19 at 20:49
15
$\begingroup$

McFadden's $R^2$ is defined as $1 - LL_{mod} / LL_0$, where $LL_{mod}$ is the log likelihood value for the fitted model and $LL_0$ is the log likelihood for the null model which includes only an intercept as predictor (so that every individual is predicted the same probability of 'success').

For a logistic regression model the log likelihood value is always negative (because the likelihood contribution from each observation is a probability between 0 and 1). If your model doesn't really predict the outcome better than the null model, $LL_{mod}$ will not be much larger than $LL_0$, and so $LL_{mod} / LL_0 \approx 1$, and McFadden's pseudo-$R^2$ is close to 0 (your model has no predictive value).

Conversely if your model was really good, those individuals with a success (1) outcome would have a fitted probability close to 1, and vice versa for those with a failure (0) outcome. In this case if you go through the likelihood calculation the likelihood contribution from each individual for your model will be close to zero, such that $LL_{mod}$ is close to zero, and McFadden's pseudo-$R^2$ squared is close to 1, indicating very good predictive ability.

As to what can be considered a good value, my personal view is that like that similar questions in statistics (e.g. what constitutes a large correlation?), is that can never be a definitive answer. Last year I wrote a blog post about McFadden's $R^2$ in logistic regression, which has some further simulation illustrations.

| cite | improve this answer | |
$\endgroup$
5
$\begingroup$

I did some more focused research on this topic, and I found that interpretations of McFadden's pseudo $R^2$ (also known as likelihood-ratio index) are not clear; however, it can range from 0 to 1, but will never reach or exceed 1 as a result of its calculation.

A rule of thumb that I found to be quite helpful is that a McFadden's pseudo $R^2$ ranging from 0.2 to 0.4 indicates very good model fit. As such, the model mentioned above with a McFadden's pseudo $R^2$ of 0.192 is likely not a terrible model, at least by this metric, but it isn't particularly strong either.

It is also important to note that McFadden's pseudo $R^2$ is best used to compare different specifications of the same model (i.e. nested models). In reference to the aforementioned example, the 6 variable model (McFadden’s pseudo $R^2$ = 0.192) fits the data better than the 5 variable model (McFadden’s pseudo $R^2$ = 0.131), which I formally tested using a log-likelihood ratio test, which indicates there is a significant difference (p < 0.001) between the two models, and thus the 6 variable model is preferred for the given data set.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ What is the reference you found which claims McFadden's R2 between 0.2 - 0.4 is a "very good" fit? $\endgroup$ – Chris May 1 '14 at 19:50
  • $\begingroup$ Btw...here is a reference and link to the original McFadden article where he defines his pseudo-R2 measure. McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. elsa.berkeley.edu/reprints/mcfadden/zarembka.pdf‎ $\endgroup$ – Chris May 1 '14 at 19:52
  • 1
    $\begingroup$ Thanks for the references. It appears that a lot of McFadden's work can be found on his Berkeley website. Below is a link to the entire book you cite above: elsa.berkeley.edu/users/mcfadden/travel.html All chapters appear as PDF. Rho-square (McFadden's pseudo R2) is mentioned in Chapter 5. Pages 122 onwards (see equation 5.33 and the graph which follows immediately thereafter). I don't see any mention of 0.2-0.4 = "VG model fit". I will keep searching for the seminal appearance of this "rule of thumb". Thanks for your help! $\endgroup$ – Chris May 5 '14 at 12:31
  • 1
    $\begingroup$ No problem! I appreciate your curiosity and thoroughness. The exact phrase can be found at lifesciencesite.com/lsj/life1002/…, where the authors state "A goodness-of-fit using McFadden‟s pseudo r-square (ρ2) is used for fitting the overall model. McFadden suggested ρ2 values of between 0.2 and 0.4 should be taken to represent a very good fit of the model (Louviere et al., 2000)." $\endgroup$ – Matt Reichenbach May 5 '14 at 13:12
  • 4
    $\begingroup$ My institution has an electronic copy of Louviere et al (2000). "Stated Choice Methods: Analysis and Applications". Cambridge University Press. This is the reference that Lee (Life Science Journal) cites for rho-squared in {0.2-0.4} = "VG fit". On page 55 of Louviere (associated with equation 3.32) we see the following quote: "Values of rho-squared between 0.2-0.4 are considered to be indicative of extremely good model fits. Simulations by Domenich and McFadden (1975) equivalence this range to 0.7 to 0.9 for a linear function". $\endgroup$ – Chris May 5 '14 at 14:19
4
$\begingroup$

In case anyone is still interested in finding McFadden's own word, here is the link. In a footnote, McFadden (1977, p.35) wrote that "values of .2 to .4 for [$\rho^2$] represent an excellent fit." The paper is available online.

http://cowles.yale.edu/sites/default/files/files/pub/d04/d0474.pdf

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.