0
$\begingroup$

I am running a logistic regression on STATA with binary response variable, and 2 predictor variable that are discrete, as such one is in % (but takes only 2 values strictly i.e., 5% or 10%) and another takes value strictly from 1 to 10. My goal is to identify the relationship between these predictor variables and success of binary variable (occurrence of event, where y=1). I also have control variables I would like to add which are either discrete, continuous or categorical. Since I am new to running a logistic regression, I have some queries.

I understand that the loglikelihood is always negative (the statistic provided when you run a regression on stata), and that larger values i.e. farther from zero (-100 as opposed to -50) is an indication of poor fit of the model. " Since the log likelihood is negative, −2LL is positive, and larger values indicate worse prediction of the dependent variable". My question is how to determine whether log likelihood is too large? Is it dependent on number of observations.

I have over 300k observations, and I am trying to ascertain if the primary predictor variable (in %)decrease the probability y =1 . In this case, how large of log likelihood is considered as too large, as in a poor fit of the model. My estimated log likelihood from only running regression with predictor variables (excluding control variables) is -207079.88. Is this too large for my dataset? My LR chi square is significant and so are the coefficients of predictor variables.


    Default | Coefficient  Std. err.      z    P>|z|  
         x1 |   4.941906   .1605616    30.78   0.000     
         x2 |  -.1427967   .0059592   -23.96   0.000    
      _cons |   .0465623   .0433171     1.07   0.282    

----------------+---------------------------------------------------------------- --------------------

Logistic regression Number of obs = 312,748 LR chi2(2) = 1768.93 Prob > chi2 = 0.0000 Log likelihood = -207079.88 Pseudo R2 = 0.0043

I understand the Pseudo r2 is too low, and preferred Mc fadden r2 minimum range is 0.2. However, I assume it will be better when I add control variables

Could someone advise when the value of log likelihood matters? Is a significant LR chi sq and significant coefficient sufficient to conclude the relationship between dependent variable and independent variable?

$\endgroup$
6
  • 1
    $\begingroup$ I'm curious where you read that the log-likelihood is an absolute measure of model fit and can have values that are too large. I have never heard of this. Where did you read that the R2 needs to be high? $\endgroup$
    – Noah
    Sep 30, 2022 at 6:46
  • $\begingroup$ @Noah "Close parallels to F and R2 exist for the logistic regression model. Just as the sum of squared errors is the criterion for selecting parameters in the linear regression model, the log likelihood is the criterion for selecting parameters in the logistic regression model. Some software presents information on the log likelihood itself; however, other software presents not the log likelihood itself, but the log likelihood multiplied by −2, which forms the basis of calculating χ2 statistics based on the likelihood ratio" - continued in the next comment $\endgroup$
    – LKho
    Sep 30, 2022 at 7:36
  • $\begingroup$ @Noah"For convenience, the log-likelihood multiplied by −2 will be abbreviated as −2LL in general, with subscripts to identify specific −2LL statistics. Since the log likelihood is negative, −2LL is positive, and larger values indicate worse prediction of the dependent variable". Source: Chapter 3: Quantitative Approaches to Model Fit and Explained Variation in Logistic Regression: From Introductory to Advanced Concepts and Applications by Scott Menard. $\endgroup$
    – LKho
    Sep 30, 2022 at 7:38
  • $\begingroup$ @Noah Since he mentioned log likelihood is a close parallel to R2 - which is a measure of goodness of fit model, i thought he implied the same for log likelihood $\endgroup$
    – LKho
    Sep 30, 2022 at 7:39
  • $\begingroup$ @Noah Regarding your second comment about R2, McFadden states in his book Behvioural Travel Modelling "while the R2 index is a more familiar concept to planner who are experienced in OLS, it is not as well behaved as the ρ2 measure, for ML estimation. Those unfamiliar with ρ2 should be forewarned that its values tend to be considerably lower than those of the R2 index...For example, *values of 0.2 to 0.4 for ρ2 represent EXCELLENT fit." Since, my R2 square value was only 0.0043, it was not a good fit. $\endgroup$
    – LKho
    Sep 30, 2022 at 7:45

1 Answer 1

1
$\begingroup$

The fact that you are referring to "good fit", "not good fit", and "excellent fit", is already an indication that such statements aren't objective; good, not good, and excellent are fundamentally subjective terms. It is a judgment call whether you think your model is good enough. It is a judgement call by your audience whether they agree with your judgment. It critically depends on what you are studying: Some things are inherently less random than others. As a consequence some things need a better fitting model before they can be considered good enough than others. It also depends on the goal of the study: if you want to use it to predict the future then you need a better fitting model than if you want a description of the association between two variables.

So you don't need a good fitting model to look at associations, especially if you are looking at human behavior and only look at few variables. There are a great many things that influence humans, so you would expect a model with two explanatory variable to capture only a tiny part of the variation among humans. That does not have to be bad, as long as your model does capture the variation you are interested in. It is good to stay humble and recognize that your study only explains a tiny amount of human behavior (and I find it highly reassuring that you cannot predict human behavior with only two variables...). However, if your research question was how variable x is associated with variable y, then your regression coefficient is your answer, regardless of how good or bad the model fit is. It does not matter that there are also many other things influencing y, that is even expected.

Also look at other studies who have studied the same phenomenon, and see what kind of fit is normal for that application. That is way more useful than some arbitrary set of cutoffs.

$\endgroup$
1
  • $\begingroup$ +1 That last paragraph is an outstanding idea! $\endgroup$
    – Dave
    Sep 30, 2022 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.