Logistic Regression: How to call the output

Question

I'm currently working on a scientific paper and I'm struggle to answer the following question: What is the right term for the output (ranging from 0 to 1) of a logistic regression?

Neither of these works (since being already taken): Certainty, probability, confidence.

Ideas: rank, probability of occurrence, probability of success..?

Answer 1

It simply is probability, you can call it "predicted" as suggested by others.

I see from the discussion that you disagree with such name, so let me proove you that this is probability.

First, recall that if $X$ is a Bernoulli distributed random variable parametrized by $p$, then $E(X) = p$. Second, take an intercept-only logistic regression model, such model will calculate mean of your predicted $Y$ variable. This would be the same as if you calculated it simply taking $\hat y_i = (1/N) \sum_{i=1}^N y_i$. This mean would converge to expected value as $N\rightarrow\infty$, i.e. to $E(Y)= p$. In fact, sample mean is a maximum likelihood estimator of $p$ for Bernoulli distributed random variable. In case of more complicated logistic regression model you predict conditional means, i.e. conditional probabilities.

Check also Why isn't Logistic Regression called Logistic Classification?

If this still does not convince you, below you can see simple R example showing exactly that case:

set.seed(123)
p1 <- 0.75
Y1 <- sample(0:1, 500, replace = TRUE, prob = c(1-p1, p1))

fit1 <- glm(Y1~1, family = "binomial")

p1
## [1] 0.75

fitted(fit1)[1] # only the first one since all predictions are the same
##     1 
## 0.762

mean(Y1)
## [1] 0.762

q <- 0.3
p2 <- c(0.4, 0.7)
X <- sample(0:1, 500, replace = TRUE, prob = c(1-q, q))
Y2 <- numeric(500)
Y2[X==0] <- sample(0:1, sum(X==0), replace = TRUE, prob = c(1-p2[1], p2[1]))
Y2[X==1] <- sample(0:1, sum(X==1), replace = TRUE, prob = c(1-p2[2], p2[2]))

fit2 <- glm(Y2~X, family = "binomial")

# predicted probabilities vs the true ones
table( ifelse(X==0, p2[1], p2[2]), round(fitted(fit2), 3)) 
##      
##      0.359 0.658
##  0.4   348     0
##  0.7     0   152

# empirical conditional probabilities (conditional means)
tapply( Y2, X, mean )
##         0         1 
## 0.3591954 0.6578947 

Answer 2

Hand-Waving: The logistic regression is some sort of squeezing a linear regression (fitting a line through you samples) into the range [0,1]. Since you have now some numerical value in the [0,1] range you can actually cal it (TADAM!) probability!

More Rigorous: What you get is $P(y=white|X)$ or in other words the probability of some sample $y$ to be labeled as $white$ (or whatever two classes that you have) depending on $X$-the data, or more specifically the independent measures of the features represented by $y$.

So to sum it up, $logit(y)$ is the probability of $y$ being ${0,1}$. If it is only about semantics, you can always drop likelihood inside.

Answer 3

If $x$ are your independent variables then the output of the logistic regression, let's call it $l(x)$, is the predicted ''conditional'' probability , i.e. the predicted value of the probability conditional on that value of $x$.

So if you have a very large number of subjects with the same value for $x$ then it is expected that a fraction $l(x)$ of these will be 'successes' (or 1).