1
$\begingroup$

I'm currently working on a scientific paper and I'm struggle to answer the following question: What is the right term for the output (ranging from 0 to 1) of a logistic regression?

Neither of these works (since being already taken): Certainty, probability, confidence.

Ideas: rank, probability of occurrence, probability of success..?

$\endgroup$
  • 3
    $\begingroup$ "Fitted probability" or "predicted probability". $\endgroup$ – markseeto Aug 3 '16 at 7:38
  • 2
    $\begingroup$ Rank is certainly wrong. That refers to first, second, third, etc., which does not correspond to the predicted probability you get from a logistic regression model. $\endgroup$ – Maarten Buis Aug 3 '16 at 9:52
4
$\begingroup$

It simply is probability, you can call it "predicted" as suggested by others.

I see from the discussion that you disagree with such name, so let me proove you that this is probability.

First, recall that if $X$ is a Bernoulli distributed random variable parametrized by $p$, then $E(X) = p$. Second, take an intercept-only logistic regression model, such model will calculate mean of your predicted $Y$ variable. This would be the same as if you calculated it simply taking $\hat y_i = (1/N) \sum_{i=1}^N y_i$. This mean would converge to expected value as $N\rightarrow\infty$, i.e. to $E(Y)= p$. In fact, sample mean is a maximum likelihood estimator of $p$ for Bernoulli distributed random variable. In case of more complicated logistic regression model you predict conditional means, i.e. conditional probabilities.

Check also Why isn't Logistic Regression called Logistic Classification?

If this still does not convince you, below you can see simple R example showing exactly that case:

set.seed(123)
p1 <- 0.75
Y1 <- sample(0:1, 500, replace = TRUE, prob = c(1-p1, p1))

fit1 <- glm(Y1~1, family = "binomial")

p1
## [1] 0.75

fitted(fit1)[1] # only the first one since all predictions are the same
##     1 
## 0.762

mean(Y1)
## [1] 0.762

q <- 0.3
p2 <- c(0.4, 0.7)
X <- sample(0:1, 500, replace = TRUE, prob = c(1-q, q))
Y2 <- numeric(500)
Y2[X==0] <- sample(0:1, sum(X==0), replace = TRUE, prob = c(1-p2[1], p2[1]))
Y2[X==1] <- sample(0:1, sum(X==1), replace = TRUE, prob = c(1-p2[2], p2[2]))

fit2 <- glm(Y2~X, family = "binomial")

# predicted probabilities vs the true ones
table( ifelse(X==0, p2[1], p2[2]), round(fitted(fit2), 3)) 
##      
##      0.359 0.658
##  0.4   348     0
##  0.7     0   152

# empirical conditional probabilities (conditional means)
tapply( Y2, X, mean )
##         0         1 
## 0.3591954 0.6578947 
$\endgroup$
  • $\begingroup$ Thank you very much for the (convincing) code! The table convinced me to go for "predicted probability" ! $\endgroup$ – Boern Aug 4 '16 at 9:32
1
$\begingroup$

Hand-Waving: The logistic regression is some sort of squeezing a linear regression (fitting a line through you samples) into the range [0,1]. Since you have now some numerical value in the [0,1] range you can actually cal it (TADAM!) probability!

More Rigorous: What you get is $P(y=white|X)$ or in other words the probability of some sample $y$ to be labeled as $white$ (or whatever two classes that you have) depending on $X$-the data, or more specifically the independent measures of the features represented by $y$.

So to sum it up, $logit(y)$ is the probability of $y$ being ${0,1}$. If it is only about semantics, you can always drop likelihood inside.

$\endgroup$
  • $\begingroup$ I think "probability" doesn't work since the result is gained for a logistic function (sigma) which has nothing to do with the "probability" - neither the bayesian nor the probabilistic one. Same applies for the "likelihood" which is occupied by by the bayesian likelihood (resulting into a posterior distribution in combination with priori knowledge). $\endgroup$ – Boern Aug 3 '16 at 8:52
  • 1
    $\begingroup$ it's not about the logit function as itself, but as using the logit as generalized linear model to fit the data, hence it's can definitely (and usually) describes a probability (usually plotted as a CDF) $\endgroup$ – elkbrs Aug 3 '16 at 9:15
  • 1
    $\begingroup$ You could call it the expected proportion of successes (1s), but many just call that a probability. The fact that the relationship between that expected probablity and a continuous X is assumed to follow an S-shaped curve changes nothing. $\endgroup$ – Maarten Buis Aug 3 '16 at 9:48
0
$\begingroup$

If $x$ are your independent variables then the output of the logistic regression, let's call it $l(x)$, is the predicted ''conditional'' probability , i.e. the predicted value of the probability conditional on that value of $x$.

So if you have a very large number of subjects with the same value for $x$ then it is expected that a fraction $l(x)$ of these will be 'successes' (or 1).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.