# Logistic regression "probability" function (is not a valid pdf...)

The idea behind logistic regression is to estimate the posterior class conditional probability, given observation x for class C_k, with a sigmoid f(C_k| x)=1/(1+exp(-w*x)) to compute the weights vector w.

In every book I've read (e.g., Bishop's PRML) f(C_k| x) is a probability density function but this is definitely not a valid pdf since the integral from minus infinity to infinity does not equal to 1 (nor it could be by any normalization since the integral is infinite).

Appreciate any explanations in this matter

• stats.stackexchange.com/questions/69820 and stats.stackexchange.com/questions/91473 look like they might answer this question. Another approach is to ask the same question for ordinary least squares regression. Now the response density is $$f(y\mid x)=\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{1}{2\sigma^2}(y-\alpha-\beta x)^2\right).$$ Although you can integrate this over all $x,$ you usually won't get $1$ as the answer. The problem is that this integral is unrelated to the regression, because it is taking some kind of average over the regressor $x.$
– whuber
Commented Jan 3, 2022 at 21:21
• $f(C_k|X=x)$ is a probability mass function for the classes conditional on $X = x$, so the joint distribution of $(X, C_k)$ would be $f(x)f(C_k|x)$, where $f(x)$ is the pdf or pmf of $X$. If you summed over the classes and integrated (summed) on $X$ the function $f(x)f(C_k|x)$, you should get $1$, is that not correct? In logistic regression, we usually do not care about $f(x)$ since we take $X$ as fixed, so we only model $f(C_k|X)$. Commented Jan 3, 2022 at 21:44
• The logistic function that you give is a distribution function, not a density function. The distribution ranges from 0 to 1. The logistic density is the derivative of the function you give. Commented Jan 4, 2022 at 0:24