Why would we use odds instead of probabilities when performing logistic regression?
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
The advantage is that the odds defined on $(0,\infty)$ map to log-odds on $(-\infty, \infty)$, while this is not the case of probabilities. As a result, you can use regression equations like $$\log \left(\frac{p_i}{1-p_i}\right) = \beta_0 + \sum_{j=1}^J \beta_j x_{ij}$$ for the log-odds without any problem (i.e. for any value of the regression coefficients and covariates a valid value for the odds are predicted). You would need extremely complicated multi-dimensional constraints on the regression coefficients $\beta_0,\beta_1,\ldots$, if you wanted to do the same for the log probability (and of course this would not work in a straightforward way for the untransformed probability or odds, either). As a consequence you get effects like being unable to have a constant risk ratio across all baseline probabilities (some risk ratios would result in probabilities > 1), while this is not an issue with an odds-ratio.
-
2$\begingroup$ I would just add to this excellent answer that with logged probabilities the maximum value can be log(1)=0. Whereas with logged odds we need not be bound to that. e.g. log (0.99/(1-0.99)) would well exceed 0. I know this point has been raised in the answer but I thought illustrating with an example would help novices such as myself $\endgroup$ Commented Dec 4, 2020 at 1:43
$\begingroup$
$\endgroup$
The odds is the expected number of "successes" per "failure", so it can take values less than one, one or more than one, but negative values won't make sense; you can have 3 successes per failure, but -3 successes per failure does not make sense. The logarithm of an odds can take any positive or negative value. Logistic regression is a linear model for the log(odds). This works because the log(odds) can take any positive or negative number, so a linear model won't lead to impossible predictions. We can do a linear model for the probability, a linear probability model, but that can lead to impossible predictions as a probability must remain between 0 and 1.
$\begingroup$
$\endgroup$
McCullagh and Nelder (1989 Generalized Linear Models) list 2 reasons.
First, analytic results with odds are more easily interpreted: the effect of a unit change in explanatory variable x2 is to increase the odds of a positive response multiplicatively by the factor exp(beta_2). Beta_x2 has units of odds/unit of x2 where x2 is continuous. Beta_x2 is the odds ratio for a categorical variable x2. The corresponding statements from the probability scale functions are more complicated. Probabilities are readily back-calculated from odds: p = (odds)/(1+odds).
Second, an important property of the logistic (log odds) function not shared by the
probability scale functions (probit, log-log) is that differences on the logistic scale can be estimated regardless of whether the data are sampled prospectively or retrospectively.
This is an advantage in medical applications because prospective studies can take years to accumulate sufficient data for making inferences.
