This is an interview question my friend asked me since she messed up with it. To be more specific,

What is the resulting distribution when a Gaussian $\mathcal{N}(\mu, \sigma^2)$ is transformed by Sigmoid $t(x)=\frac{1}{1+e^{-x}}$.

I tried to derive and got the following $$ \begin{aligned} \log y&=x-\log (1+e^x)\\ x&=\frac{1}{\sqrt{2\pi\sigma}}\exp(-\frac{(x-\mu)^2}{2\sigma^2}) \end{aligned} $$ But the $\log(\cdot)$ seems to be intractable when I plug $x$ in.

I also tried to simulate this process and got

enter image description here

It seems that the resulting distribution is still Gaussian (with some elevation and shrinking in magnitude). But I still would like to know the theoretical derivation.

Could someone help me and my friend? Thank you in advance.

  • $\begingroup$ 1. Your question (specifically the quoted part) uses the same symbol ($\sigma$) for two entirely different things. This will likely lead to disaster. 2. By "passed through" I take it you mean "transformed by", i.e. if $X\sim N(\mu,\sigma^2)$ you're seeking the density of $Y=t(X)$ where $t(x) = 1/[1+\exp(-x)]$? $\endgroup$ – Glen_b -Reinstate Monica Nov 8 '19 at 4:43
  • $\begingroup$ @Glen_b Sorry about the confusion. Will update the question. $\endgroup$ – Mr.Robot Nov 8 '19 at 4:47
  • $\begingroup$ Do you seek methodology (how do I compute the density of a transformed random variable?) or just the answer -- it's (a well known distribution with its own wikipedia page and many questions about it on site). $\endgroup$ – Glen_b -Reinstate Monica Nov 8 '19 at 4:49
  • $\begingroup$ @Glen_b I didn't know this. Could you provide a pointer to this page. Also, it would be great if you could help me with the derivation. $\endgroup$ – Mr.Robot Nov 8 '19 at 4:52
  • 1
    $\begingroup$ 1. An example of a previous question is here: Distribution of a logit transform of a normal random variable 2. Wikipedia on How to transform random variables 3. Wikipedia on the resulting distribution, the logit-normal $\endgroup$ – Glen_b -Reinstate Monica Nov 8 '19 at 5:05

Hint: Let $Y$ be the random variable obtained by taking a sigmoid of a normal random variable, $X$.

This means that $\operatorname{logit}(Y)$ is normal. We might call this a ______ normal random variable.

Here is a another hint: If $\log(Y)$ is normal, we call $Y$ a log normal distribution. Can you apply the same naming scheme to your problem?

Furthermore, you've just transformed the gaussian density in your example. Try drawing from a normal and then applying the sigmoid to the draws. Then, plot a histogram.


Not the answer you're looking for? Browse other questions tagged or ask your own question.