7
$\begingroup$

Is Wikipedia's page on the sigmoid function incorrect?

It states that:

A common example of a sigmoid function is the logistic function

From my knowledge of machine learning, I thought that "the sigmoid function" is defined as the logistic function, $$\sigma(z) = \frac {1} {\left(1 + e^{-z}\right)}\text{.}$$ I have never seen or heard the phrasing that the logistic function is a type of sigmoid function.

Furthermore, that Wikipedia page says that other examples of a sigmoid function are the tanh and arctan functions. Again, I've never seen tanh nor arctan described as a type of sigmoid function.

These functions are considered to be peers, usually in a context like:

We can use various non-linear functions in this neural network, such as the sigmoid, tanh, and ReLU activation functions.

What am I missing here? Is the Wikipedia article correct or incorrect? I find that Wikipedia is usually accurate for math terms.

$\endgroup$
6
  • 9
    $\begingroup$ tanh, if you scale and shift domain and range, is the same as the logistic function. $\endgroup$ yesterday
  • 1
    $\begingroup$ Why was this question closed? My question and the linked question are different! $\endgroup$ 17 hours ago
  • 1
    $\begingroup$ That is probably because the tanh switches from -1 to 1, while the logistic switches from 0 to 1. Thus the mentioned transformation of the range. $\endgroup$ 16 hours ago
  • 1
    $\begingroup$ @LutzLehmann: If you need to transform the tanh to get the same output as the logistic function, then tanh is not the logistic function and should not be mentioned together except as comparable peers. $\endgroup$ 15 hours ago
  • 7
    $\begingroup$ Usually, one applies linear transformations before and after the sigmoid function. In that context there is no difference to using tanh, only the parameters of the linear transforms shift somewhat. $\endgroup$ 15 hours ago
36
$\begingroup$

The unsatisfying answer is "It depends who you ask." "Sigmoid", if you break it into parts, just means "S-shaped".

The logistic sigmoid function is so prevalent that people tend to gloss over the word "logistic". For machine learning folks, it's become the exemplar of the class, and most call it the sigmoid function. (Is it myopia to call it the sigmoid function?) Still, there are other communities that use S-shaped functions.

$\endgroup$
12
  • 11
    $\begingroup$ I'm analytical chemist, and we use "sigmoid" in the more general S-shape sense without implying what function exactly. E.g. the rather typical detector behaviour that you get some roughly constant signal at very low concentrations, then the signal increases with analyte concentration (that's what we want to use) and finally, at high concentrations the signal becomes constant again, e.g. because of detector saturation is called sigmoid. $\endgroup$ 22 hours ago
  • 14
    $\begingroup$ Re "is it myopia?": yes, definitely. This function has been around--with an established name ("logistic")--since the mid-1800's. A community that creates a new name for such an old, well-known object is actively rejecting its intellectual history. $\endgroup$
    – whuber
    17 hours ago
  • 10
    $\begingroup$ Intellectual history does matter. Those who don't know it are doomed to repeat it, as the adage goes. It is difficult (practically impossible) to acquire a deep understanding of a concept or technique if you have to repeat for yourself centuries of investigation and discovery. In the present case, anybody who has taken a freshman college course in math, chemistry, physics, or biology has learned about logistic functions under that name, so ignorance is no excuse. Even Isaac Newton acknowledged that he "stood on the shoulders of giants." We, too, should take advantage of what precedes us. $\endgroup$
    – whuber
    15 hours ago
  • 15
    $\begingroup$ My point is that ML people thinking that other (older) disciplines' names are wrong is hubris. (I say this as an ML person.) Just because we're a community with a large, loud online presence doesn't mean that we're the only truth out there. Norms in your field ≠ norms in other fields. So while your colleagues may understand "sigmoid function" to mean the logistic sigmoid function specifically, analytical chemists are also well-grounded in calling a broader class of functions "sigmoid functions". $\endgroup$ 15 hours ago
  • 8
    $\begingroup$ @stackoverflowuser2010 There are lots of examples of machine learning/neural networks folks redefining terms. For instance, I know that when a NN paper writes about "cross entropy loss," they're almost certainly referring to "categorical cross entropy," even though you can write a cross entropy loss for other distributions. $\endgroup$
    – Sycorax
    14 hours ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.