1
$\begingroup$

Consider the probability density function: $$ f(x) = \frac{\beta_1*exp(\beta_0+x\beta_1)}{(exp(\beta_0+x\beta_1) + 1)^2} $$

This is a valid pdf for any $\beta_0$ and $\beta_1 > 0$ when x has support on the whole number line.

My question is, is this a named distribution (possible reparameterized) who's properties have already been derived?

If it helps: $$ E[X] = -\frac{\beta_0}{\beta_1} $$

It's also a symmetric distribution with fat tails (fatter than the normal).

My derivation of this distribution was considering a logistic regression of a single variable (e.g. time to death) on survival probability. If $\beta_1$ is positive, than the fitted logistic regression curve has all the properties of a CDF, with the above pdf being the derivative of the fitted line. It seems to me that knowing the properties of this distribution would help us understand the implication of logistic regression better (specifically the 'linear on the logit scale' assumption).

$\endgroup$

1 Answer 1

2
$\begingroup$

The following is under the assumption that somewhere a negative sign got lost in the mix. That is, it should be $\exp\{-(\beta_0 + \beta_1x)\}$ in the numerator and denominator. I assume this because the logistic regression curve can be formulated as $$\frac{1}{1 + \exp\{-(\beta_0 + \beta_1x)\}}.$$ And I cannot replicate your results without that assumption. Hopefully my derivative skills aren't that rusty.

Then you would in fact have a reparameterization of the Logistic distribution. I would say that's not a coincidence given you pulled this from logistic regression.

The general form of the pdf being $$f(x) = \dfrac{e^{-(x - \mu)/s}}{s(e^{-(x - \mu)/s} + 1)^2}$$

The expected value is $\mu$ (surprise...) and $s$ is the scale parameter.

Put $-(\beta_0 + x\beta_1)$ as a fraction over $\beta_1^{-1}$ and pull out a negative sign from $\beta_0$. I'm a real stickler for scale parameters being in the denominator.

$$-(\beta_0 + x\beta_1) = -\frac{\left(x - \left(-\frac{\beta_0}{\beta_1}\right)\right)}{\beta_1^{-1}}$$

With that we can get the pdf for $x$ being a random variable following a logistic distribution with $\mu = -\beta_0/\beta_1$ and $s = \beta_1^{-1}$ when we assume $\beta_1 > 0$

$$f(x) = \dfrac{\exp\left\{-\frac{\left(x - \left(-\frac{\beta_0}{\beta_1}\right)\right)}{\beta_1^{-1}}\right\}}{\beta_1^{-1}\left(\exp\left\{-\frac{\left(x - \left(-\frac{\beta_0}{\beta_1}\right)\right)}{\beta_1^{-1}}\right\} + 1\right)^2}$$

=========================================================================

Edit in response to @jntrcs' question: "If I start with log(y/(1-y)) = B_0 +xB_1 how do I end up with 1 / (1+exp(-(b_0+B_1x)))? Otherwise, the logistic distribution makes a lot of sense."

Lets start simple and go from there by solving for $y$ in $\log\left(y/(1-y)\right) = x$.

$$y = \frac{e^x}{1+e^x}$$

This is the basic form of the logistic sigmoid function. Now, if you want to futz around with it (like take a derivative) it's a bit messy and we can clean it up quite easily by multiplying the numerator and denominator by $e^{-x}$. This cancels out the $e^x$ in the numerator and denominator leaving us with

$$\frac{1}{e^{-x} + 1},$$

which I would say is much nicer and easier to deal with. This also just so happens to be the standard logistic ($\mu = 0$, and $s = 1$) cumulative distribution function.

Taking the linear form using the logit link of $\log\left(\frac{y}{1-y}\right) = \beta_0 +\beta_1x$ gives us

$$y = \dfrac{exp\{\beta_0 + \beta_1x\}}{1 + \exp\{\beta_0 + \beta_1x\}},$$

and similarly to the simpler example, multiply through by $\exp\{-(\beta_0 + \beta_1x)\}$ to get

$$\frac{1}{1 + \exp\{-(\beta_0 + \beta_1x)\}}.$$

Then fool around with it like I had mentioned earlier to get the logistic distribution form.

As far as your investigation into the linearity and logit assumption... I wish I could give some insight there. In practice, the mentality is generally going to be $crossyourfingersandhopeitsokay$. If you want to think about it more abstractly, you can use the fact that logistic regression transfers the model into hyperbolic geometry.

  • CDF/Logistic Regression Curve: $\dfrac{1}{1 + e^{-x}} = \frac{1}{2} + \frac{1}{2}\tanh\left(x/2\right)$

  • pdf: $\dfrac{e^{-x}}{(1 + e^{-x})^2} = \frac{1}{4}\mathrm{sech}^2\left(x/2\right)$

I don't really know how illuminating that is. Hyperbolic geometry is kind of funky until you get used to it.

Another thing to consider is that formally, $\beta_1$ needs to be positive to for the pdf to be valid. However, I would consider the fact that $\beta_1$ is a sort of scale parameter in the regression and ignoring the sign may allow for more general conclusions. Also, ignoring things in math makes things simpler and then you can cross your fingers that it translates to that whole "without loss of generality" mentality.

Latex Quirk PSA: There is \tanh command, but not a \sech command. Those poor secants always getting overlooked.

$\endgroup$
3
  • 1
    $\begingroup$ Great first answer! Welcome to the site. $\endgroup$
    – Noah
    Dec 23, 2020 at 7:52
  • 1
    $\begingroup$ If I start with log(y/(1-y)) = B_0 +x*B_1 how do I end up with 1 / (1+exp(-(b_0+B_1*x)))? Otherwise, the logistic distribution makes a lot of sense. $\endgroup$
    – jntrcs
    Dec 23, 2020 at 16:20
  • $\begingroup$ Forgot to mention I added clarification for your question. $\endgroup$ Dec 23, 2020 at 21:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.