Why do we use the natural exponential in logistic regression? I would like to intuitively understand the benefit of using the natural exponential in the sigmoid function used in logistic regression.
Why should it have to be $e^x$ instead of, for example $2^x$?
 A: Because base $e$ is convenient, and it doesn't matter if you can freely scale your coefficient estimate.
Would using a functional form of $\frac{a^\mathbf{x\cdot b}}{1 + a^\mathbf{x\cdot b} }$ change your explanatory power? No.
Explanation:
I gave basically the same answer here for the softmax function.
Observe that $  e^ {  \mathbf{x} \cdot \mathbf{b} \left( \ln a \right) } =  a^ {\mathbf{x} \cdot \mathbf{b}}$. Hence:
$$ \frac{a^\mathbf{x\cdot b}}{1 + a^\mathbf{x\cdot b} } = \frac{e^\mathbf{x\cdot \tilde{b}}}{1 + e^\mathbf{x\cdot \tilde{b}} } $$
Where $\tilde{\mathbf{b}} = \left( \ln a \right) \mathbf{b} $. So using a different base than $e$ in the sigmoid function is the same as scaling your $\mathbf{b}$ vector. 
A: In binary regression, one can use any cdf to relate the probability $\mathbb{P}(Y=1|\mathbf{x})$ and $\mathbf{x}$ in a generalised linear way
$$\mathbb{P}(Y=1|\mathbf{x})=\Phi(\mathbf{x}^\text{T}\beta)$$as in


*

*logistic cdf, $\Phi(t)=1/\{1+1/e^t\}$

*probit (Normal) cdf, $\Phi(t)=\int_{-\infty}^t \varphi(x)\text{d}x$

*log-log cdf, $\Phi(t)=\exp\{-\exp(-x)\}$


The logistic offers some advantages, as making the conditional regression an exponential family model.
A: For a Bernoulli likelihood, the variance is a function of the mean such that:
$$\text{var}(Y) = E(Y)(1-E(Y))$$
It turns out that a sigmoid function, also called the "inverse link" (for a logistic regression) function: $S(x) = \frac{\exp(x)}{1+\exp(x)}$ has the property that:
$$\frac{\partial}{\partial x} S(X) = S(X)(1-S(X))$$
It turns out this property holds for all GLMs using canonical parametrizations for exponential families.
