49
$\begingroup$

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this pseudo R-squared?

Is it a relative comparison for nested models (e.g. a 6 variable model has a McFadden's pseudo R-squared of 0.192, whereas a 5 variable model (after removing one variable from the aforementioned 6 variable model), this 5 variable model has a pseudo R-squared of 0.131. Would we would want to keep that 6th variable in the model?) or is it an absolute quantity (e.g. a given model that has a McFadden's pseudo R-squared of 0.192 is better than any existing model with a McFadden's pseudo R-squared of 0.180 (for even non-nested models)? These are just possible ways to look at McFadden’s pseudo R-squared; however, I assume these two views are way off, thus the reason why I am asking this question here.

I have done a great deal of research on this topic, and I have yet to find the answer that I am looking for in terms of being able to interpret a McFadden's pseudo R-squared of 0.192. Any insight and/or references are greatly appreciated! Before answering this question, I am aware that this isn't the best measure to describe a logistic regression model, but I would like to have a greater understanding of this statistic regardless!

$\endgroup$

5 Answers 5

65
$\begingroup$

So I figured I'd sum up what I've learned about McFadden's pseudo $R^2$ as a proper answer.

The seminal reference that I can see for McFadden's pseudo $R^2$ is: McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. http://eml.berkeley.edu/~mcfadden/travel.html Figure 5.5 shows the relationship between $\rho^2$ and traditional $R^2$ measures from OLS. My interpretation is that larger values of $\rho^2$ (McFadden's pseudo $R^2$) are better than smaller ones.

The interpretation of McFadden's pseudo $R^2$ between 0.2-0.4 comes from a book chapter he contributed to: Bahvioural Travel Modelling. Edited by David Hensher and Peter Stopher. 1979. McFadden contributed Ch. 15 "Quantitative Methods for Analyzing Travel Behaviour on Individuals: Some Recent Developments". Discussion of model evaluation (in the context of multinomial logit models) begins on page 306 where he introduces $\rho^2$ (McFadden's pseudo $R^2$). McFadden states "while the $R^2$ index is a more familiar concept to planner who are experienced in OLS, it is not as well behaved as the $\rho^2$ measure, for ML estimation. Those unfamiliar with $\rho^2$ should be forewarned that its values tend to be considerably lower than those of the $R^2$ index...For example, values of 0.2 to 0.4 for $\rho^2$ represent EXCELLENT fit."

So basically, $\rho^2$ can be interpreted like $R^2$, but don't expect it to be as big. And values from 0.2-0.4 indicate (in McFadden's words) excellent model fit.

$\endgroup$
2
  • 3
    $\begingroup$ Good wrap-up, Chris. Thank you for your efforts! $\endgroup$ Commented May 21, 2014 at 21:21
  • $\begingroup$ I arrived late to the discussion, but I will leave this link where they explain the R2 MacFadden compared to other adjustment measures: statisticalhorizons.com/r2logistic $\endgroup$ Commented Dec 4, 2019 at 20:49
22
$\begingroup$

McFadden's $R^2$ is defined as $1 - LL_{mod} / LL_0$, where $LL_{mod}$ is the log likelihood value for the fitted model and $LL_0$ is the log likelihood for the null model which includes only an intercept as predictor (so that every individual is predicted the same probability of 'success').

For a logistic regression model the log likelihood value is always negative (because the likelihood contribution from each observation is a probability between 0 and 1). If your model doesn't really predict the outcome better than the null model, $LL_{mod}$ will not be much larger than $LL_0$, and so $LL_{mod} / LL_0 \approx 1$, and McFadden's pseudo-$R^2$ is close to 0 (your model has no predictive value).

Conversely if your model was really good, those individuals with a success (1) outcome would have a fitted probability close to 1, and vice versa for those with a failure (0) outcome. In this case if you go through the likelihood calculation the likelihood contribution from each individual for your model will be close to zero, such that $LL_{mod}$ is close to zero, and McFadden's pseudo-$R^2$ squared is close to 1, indicating very good predictive ability.

As to what can be considered a good value, my personal view is that like that similar questions in statistics (e.g. what constitutes a large correlation?), is that can never be a definitive answer. Last year I wrote a blog post about McFadden's $R^2$ in logistic regression, which has some further simulation illustrations.

$\endgroup$
11
$\begingroup$

I did some more focused research on this topic, and I found that interpretations of McFadden's pseudo $R^2$ (also known as likelihood-ratio index) are not clear; however, it can range from 0 to 1, but will never reach or exceed 1 as a result of its calculation.

A rule of thumb that I found to be quite helpful is that a McFadden's pseudo $R^2$ ranging from 0.2 to 0.4 indicates very good model fit. As such, the model mentioned above with a McFadden's pseudo $R^2$ of 0.192 is likely not a terrible model, at least by this metric, but it isn't particularly strong either.

It is also important to note that McFadden's pseudo $R^2$ is best used to compare different specifications of the same model (i.e. nested models). In reference to the aforementioned example, the 6 variable model (McFadden’s pseudo $R^2$ = 0.192) fits the data better than the 5 variable model (McFadden’s pseudo $R^2$ = 0.131), which I formally tested using a log-likelihood ratio test, which indicates there is a significant difference (p < 0.001) between the two models, and thus the 6 variable model is preferred for the given data set.

$\endgroup$
10
  • 1
    $\begingroup$ What is the reference you found which claims McFadden's R2 between 0.2 - 0.4 is a "very good" fit? $\endgroup$
    – Chris
    Commented May 1, 2014 at 19:50
  • 1
    $\begingroup$ Btw...here is a reference and link to the original McFadden article where he defines his pseudo-R2 measure. McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. elsa.berkeley.edu/reprints/mcfadden/zarembka.pdf‎ $\endgroup$
    – Chris
    Commented May 1, 2014 at 19:52
  • 1
    $\begingroup$ Thanks for the references. It appears that a lot of McFadden's work can be found on his Berkeley website. Below is a link to the entire book you cite above: elsa.berkeley.edu/users/mcfadden/travel.html All chapters appear as PDF. Rho-square (McFadden's pseudo R2) is mentioned in Chapter 5. Pages 122 onwards (see equation 5.33 and the graph which follows immediately thereafter). I don't see any mention of 0.2-0.4 = "VG model fit". I will keep searching for the seminal appearance of this "rule of thumb". Thanks for your help! $\endgroup$
    – Chris
    Commented May 5, 2014 at 12:31
  • 1
    $\begingroup$ No problem! I appreciate your curiosity and thoroughness. The exact phrase can be found at lifesciencesite.com/lsj/life1002/…, where the authors state "A goodness-of-fit using McFadden‟s pseudo r-square (ρ2) is used for fitting the overall model. McFadden suggested ρ2 values of between 0.2 and 0.4 should be taken to represent a very good fit of the model (Louviere et al., 2000)." $\endgroup$ Commented May 5, 2014 at 13:12
  • 4
    $\begingroup$ My institution has an electronic copy of Louviere et al (2000). "Stated Choice Methods: Analysis and Applications". Cambridge University Press. This is the reference that Lee (Life Science Journal) cites for rho-squared in {0.2-0.4} = "VG fit". On page 55 of Louviere (associated with equation 3.32) we see the following quote: "Values of rho-squared between 0.2-0.4 are considered to be indicative of extremely good model fits. Simulations by Domenich and McFadden (1975) equivalence this range to 0.7 to 0.9 for a linear function". $\endgroup$
    – Chris
    Commented May 5, 2014 at 14:19
8
$\begingroup$

In case anyone is still interested in finding McFadden's own word, here is the link. In a footnote, McFadden (1977, p.35) wrote that "values of .2 to .4 for [$\rho^2$] represent an excellent fit." The paper is available online.

http://cowles.yale.edu/sites/default/files/files/pub/d04/d0474.pdf

$\endgroup$
3
$\begingroup$

The measures summarized here have advantages over McFadden’s pseudo $R^2$. My new favorite measure is a pseudo adjusted $R^2$ that is very close to the traditional adjusted $R^2$ when the model is linear. It uses the effective sample size instead of the apparent sample size. For a binary Y the effective sample size is $3np(1-p)$ where $p$ is the proportion of Y=1. This makes the $R^2$ more realistic (larger). By also adjusting for the number of non-intercepts in the model, the pseudo adjusted $R^2$ is nearly unbiased, i.e., does not benefit from overfitting.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.