In section Sigmoid Units for Bernoulli Output Distributions of the Deep Learning book, the author gave a logistic regression model for binary classification $P(y)=\sigma(z)$ where $\sigma$ is the logistic sigmoid function and $z$ is a linear/affine transformation of input $x$. The author then said that:

Saturation thus occurs only when the model already has the right answer—when $y=1$ and $z$ is very positive, or $y=0$ and $z$ is very negative.

I'm just wondering why this is the correct input-output relationship to be learned.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.