4
$\begingroup$

I am doing logistic regression in R on a binary dependent variable with only one independent variable. I found the odd ratio as 0.99 for an outcomes. This can be shown in following. Odds ratio is defined as, $ratio_{odds}(H) = \frac{P(X=H)}{1-P(X=H)}$. As given earlier $ratio_{odds} (H) = 0.99$ which implies that $P(X=H) = 0.497$ which is close to 50% probability. This implies that the probability for having a H cases or non H cases 50% under the given condition of independent variable. This does not seem realistic from the data as only ~20% are found as H cases. Please give clarifications and proper explanations of this kind of cases in logistic regression.

I am hereby adding the results of my model output:

M1 <- glm(H~X, data=data, family=binomial())
summary(M1)

Call:
glm(formula = H ~ X, family = binomial(), data = data)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.8563   0.6310   0.6790   0.7039   0.7608  

Coefficients:
                Estimate      Std. Error      z value     Pr(>|z|)    
(Intercept)    1.6416666      0.2290133      7.168      7.59e-13 ***
   X          -0.0014039      0.0009466     -1.483      0.138    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 1101.1  on 1070  degrees of freedom
Residual deviance: 1098.9  on 1069  degrees of freedom
  (667 observations deleted due to missingness)
AIC: 1102.9

Number of Fisher Scoring iterations: 4


exp(cbind(OR=coef(M1), confint(M1)))
Waiting for profiling to be done...
                                      OR           2.5 %       97.5 %
(Intercept)                    5.1637680       3.3204509     8.155564
     X                         0.9985971       0.9967357     1.000445

I have 1738 total dataset, of which H is a dependent binomial variable. There are 19.95% fall in (H=0) category and remaining are in (H=1) category. Further this binomial dependent variable compare with the covariate X whose minimum value is 82.23, mean value is 223.8 and maximum value is 391.6. The 667 missing values correspond to the covariate X i.e 667 data for X is missing in the dataset out of 1738 data.

$\endgroup$
  • $\begingroup$ What is your interpretation of the intercept estimate? $\endgroup$ – whuber Jun 20 '16 at 17:24
  • $\begingroup$ The estimated coefficient for intercept is 1.6416666, therefore the odds of being in H=1 class i.e p(H=1), when the independent variable X exposure is zero is exp(1.6416666)=5.1637680. The odds are high,but if we look at the variable X, the minimum exposure value of X is 82.23 and mean value of X is 223.38. So the intercept in this model corresponds to the log odds of being in H=1 class or group or category when X is at the hypothetical value of zero. Any correction or suggestions are extremely welcome. $\endgroup$ – Saurabh Sinha Jun 20 '16 at 17:53
  • $\begingroup$ Doesn't that odds of 1.64 correspond to the "~20%" you mention? $\endgroup$ – whuber Jun 20 '16 at 18:21
  • $\begingroup$ No No, that 20% are those data which are corresponds to H=0 cases. But as per Odds of 0.99, the probability of H=0 class and H=1 class become almost equal i.e about 50%, which is not in the dataset as only about 20% data belong to H=0 class. Hope it will help you. $\endgroup$ – Saurabh Sinha Jun 20 '16 at 18:27
  • 3
    $\begingroup$ Please include that information in your question: most people will not read through these comments (which themselves could eventually be deleted or migrated elsewhere). In the meantime, you might find it of great interest to note that $$1/(1 + \exp(1.64166666 - 0.0014039\times 223.8)) = 20.96\%$$ is remarkably close to $19.95\%$. This is not a coincidence. $\endgroup$ – whuber Jun 20 '16 at 18:46
5
$\begingroup$

Summary

The question misinterprets the coefficients.

The software output shows that the log odds of the response don't depend appreciably on $X$, because its coefficient is small and not significant ($p=0.138$). Therefore the proportion of positive results in the data, equal to $100 - 19.95\% \approx 80\%$, ought to have a log odds close to the intercept of $1.64$. Indeed,

$$\log\left(\frac{80\%}{20\%}\right) = \log(4) \approx 1.4$$

is only about one standard error ($0.22$) away from the intercept. Everything looks consistent.


Detailed analysis

This generalized linear model supposes that the log odds of the response $H$ being $1$ when the independent variable $X$ has a particular value $x$ is some linear function of $x$,

$$\text{Log odds}(H=1\,|\,X=x) = \beta_0 + \beta_1 x.\tag{1}$$

The glm command in R estimated these unknown coefficients with values $$\hat\beta_0 = 1.641666\pm 0.2290133$$ and $$\hat\beta_1 = -0.0014039\pm 0.0009466.$$

The dataset contains a large number $n$ of observations with various values of $x$, written $x_i$ for $i=1, 2, \ldots, n$, which range from $82.3$ to $391.6$ and average $\bar x = 223.8$. Formula $(1)$ enables us to compute the estimated probabilities of each outcome, $\Pr(H=1\,|\,X=x_i)$. If the model is any good, the average of those probabilities ought to be close to the average of the outcomes.

Since the odds are, by definition, the ratio of a probability to its complement, we can use simple algebra to find the estimated probabilities in terms of the log odds

$$\widehat\Pr(H=1\,|\,X=x) = 1 - \frac{1}{1 + \exp\left(\hat\beta_0 + \hat\beta_1 x\right)}.$$

As a nonlinear function of $x$, that's difficult to average. However, provided $\beta_1 x$ is small (much less than $1$ in size) and $1+\exp(\hat\beta_0)$ is not small (it exceeds $6$ in this case), we can safely use a linear approximation

$$\frac{1}{1 + \exp\left(\hat\beta_0 + \hat\beta_1 x\right)} = \frac{1}{1 + \exp(\hat\beta_0)}\left(1 - \hat\beta_1 x \frac{\exp{\hat\beta_0}}{1 + \exp(\hat\beta_0)}\right) + O\left(\hat\beta_1 x\right)^2.$$

Since the $x_i$ never exceed $391.6$, $|\hat\beta_1 x_i|$ never exceeds $391.6\times 0.0014039 \approx 0.55$, so we're ok. Consequently, the average of the outcomes may be approximated as

$$\eqalign{ \frac{1}{n}\sum_{i=1}^n \widehat\Pr(H=1\,|\,X=x) &\approx \frac{1}{n}\sum_{i=1}^n \left(1 - \frac{1}{1 + \exp(\hat\beta_0)}\left(1 - \hat\beta_1 x_i \frac{\exp{\hat\beta_0}}{1 + \exp(\hat\beta_0)}\right)\right)\\ &= 0.162238 + 0.000190814 \bar{x} \\ &= 20.4943\%. }$$

Although that's not exactly equal to the $19.95\%$ observed in the data, it is more than close enough, because $\hat\beta_1$ has a relatively large standard error. For example, if $\beta_1$ were increased by only $0.3$ of its standard error to $-0.0011271$, then the previous calculation would produce $19.95\%$ exactly.

$\endgroup$
  • 1
    $\begingroup$ In some posts I have been critical of the use of too many significant figures in publications. In this answer I have displayed many more sig figs than necessary in order to help anyone who might be trying to reproduce the calculations. At most two significant figures of precision are supported by the information presented in the question (because we just don't know the details of how the $x_i$ are distributed). $\endgroup$ – whuber Jun 23 '16 at 15:08
  • $\begingroup$ Thanks Whuber, Can you suggest me some literature and text for better understanding on logistic regression analysis, its diagnostic, Goodness of fit and others or may be some hands on practice papers. As i am doing all this in R particularly in mgcv package. It will be very helpful for me. $\endgroup$ – Saurabh Sinha Jun 24 '16 at 5:57
  • $\begingroup$ Hosmer & Lemeshow, Applied Logistic Regression. (I am familiar with the second edition; it's now in a third edition.) $\endgroup$ – whuber Jun 24 '16 at 13:36
  • $\begingroup$ Hi @whuber, are there any assumptions of a linear relationship in logistic regression? Even in the logit link function and $X$? $\endgroup$ – Pierre Lafortune Sep 20 '16 at 19:22
  • 1
    $\begingroup$ @Pierre Logistic regression shares most of the benefits and problems of least-squares multiple regression. Many of the solutions to OLS problems also apply to logistic regression. For instance, splining the regressors can both test for and account for nonlinearities. $\endgroup$ – whuber Sep 20 '16 at 20:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.