47
$\begingroup$

I have a logit model that comes up with a number between 0 and 1 for many cases, but how can we interprete this?

Lets take a case with a logit of 0.20

Can we assert that there is 20% probability that a case belongs to group B vs group A?

is that the correct way of interpreting the logit value?

$\endgroup$
4
  • 4
    $\begingroup$ In addition to @SvenHohenstein's good answer below, it may help you to read my answer here: Interpretation of simple predictions to odds ratios in logistic regression, which contains additional basic information about probabilities & odds. Note that the logit can be understood more abstractly as a link function; you can read more about that here: difference-between-logit-and-probit-models (although this answer might be a bit more technical). $\endgroup$ Commented Mar 20, 2013 at 23:08
  • 1
    $\begingroup$ I want to know why the pronunciation of logit is neither like logarithm nor like logistic $\endgroup$
    – Henry
    Commented Nov 15, 2013 at 1:14
  • $\begingroup$ @Henry - According to Wiktionary, the U.S. pronunciation of 'logit' /ˈloʊdʒɪt/ (en.wiktionary.org/wiki/logit) is like 'logistic' (/loʊˈdʒɪs.tɪk/) (en.wiktionary.org/wiki/logistic). $\endgroup$
    – shaneb
    Commented Feb 8, 2019 at 22:39
  • 1
    $\begingroup$ @shaneb - fair enough - thought that only shifts the question to the unlogical pronounciation of logistic $\endgroup$
    – Henry
    Commented Feb 10, 2019 at 20:38

3 Answers 3

81
$\begingroup$

The logit $L$ of a probability $p$ is defined as

$$L = \ln\frac{p}{1-p}$$

The term $\frac{p}{1-p}$ is called odds. The natural logarithm of the odds is known as log-odds or logit.

The inverse function is

$$p = \frac{1}{1+e^{-L}}$$

Probabilities range from zero to one, i.e., $p\in[0,1]$, whereas logits can be any real number ($\mathbb{R}$, from minus infinity to infinity; $L\in (-\infty,\infty)$).

A probability of $0.5$ corresponds to a logit of $0$. Negative logit values indicate probabilities smaller than $0.5$, positive logits indicate probabilities greater than $0.5$. The relationship is symmetrical: Logits of $-0.2$ and $0.2$ correspond to probabilities of $0.45$ and $0.55$, respectively. Note: The absolute distance to $0.5$ is identical for both probabilities.

This graph shows the non-linear relationship between logits and probabilities:

enter image description here

The answer to your question is: There is a probability of about $0.55$ that a case belongs to group B.

$\endgroup$
4
  • $\begingroup$ " Logits of $-0.2$ and $0.2$ correspond to probabilities of $0.45$ and $0.55$ respectively." How does it imply that logit distribution is symmetric? $\endgroup$
    – user 31466
    Commented Feb 12, 2015 at 3:50
  • $\begingroup$ There is a probability of about $0.55$ that a case belong to group B. When will it belong to group A ? $\endgroup$
    – user 31466
    Commented Feb 12, 2015 at 3:55
  • 2
    $\begingroup$ @Leaf Since there are only two groups, A and B, the probability for group A is $1 - 0.55 = 0.45$. $\endgroup$ Commented Feb 12, 2015 at 6:59
  • 1
    $\begingroup$ @Here, symmetry is related to the absolute difference to a probability of $0.5$ or a logit of $0$. If probability is $0.5 + x$, the logit is $0 + y$; If probability is $0.5 - x$, the logit is $0 - y$. Here, $\text{sign}(x) = \text{sign}(y).$ $\endgroup$ Commented Feb 12, 2015 at 7:02
5
$\begingroup$

To add a more modern (but not very deep) perspective, consider how it's used in deep learning (ha, pun intended...):

logit is referred to the output of a function (e.g. a Neural Net) just before it's normalization (which we usually use the softmax). This is also known as the code. So if for label $y$ we have score $f_y(x)$ then the logit is:

$$ logit = \log \left( \frac{ e^{f_y(x)} }{Z} \right) = score = f_y(x)$$

Where $Z$ is the standard partition function. By the way, this is all over the place in the pytorch and tensorflow documentation.

So you can interpret it as:

the (unnormalized) score for a label or (functional confidence) for a specific class/label.

One of the many references: https://stackoverflow.com/questions/41455101/what-is-the-meaning-of-the-word-logits-in-tensorflow

$\endgroup$
0
$\begingroup$

Could you maybe specify your model and give a screenshot of the output, then I could give you an detailed answer, but as a first try.... you may want to check out also the following examples on these websites:

http://www.ats.ucla.edu/stat/stata/seminars/stata_logistic/default.htm

http://www.ats.ucla.edu/stat/stata/dae/logit.htm

http://www.ats.ucla.edu/stat/stata/faq/oratio.htm

http://www.ats.ucla.edu/stat/mult_pkg/faq/general/odds_ratio.htm

so if the coefficient is 0.2 it depends on the variable, I guess you have a dummy, which is e.g. 0 for group B and 1 for group A?

odds ratio is given by: $OR = e^b$

so in your case: $e^{70.20}$

This would be the odds ratio of your group variable corresponding to your reference group.

$\endgroup$
1
  • 3
    $\begingroup$ I believe the OP is asking about how to interpret logits, not how to perform logistic regression. $\endgroup$
    – whuber
    Commented Mar 20, 2013 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.