Questions tagged [sigmoid-curve]

A sigmoid function is a mathematical function having a characteristic "S"-shaped curve or sigmoid curve. Often, sigmoid function refers to a special case of the logistic function. It is closely related to the logistic regression.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Backpropagation With Sigmoid Output Function Question

I am deriving a Weight update for a simple toy network with a Sigmoid Output Layer. I need some help double checking my math to make sure I did it correctly. I am using Cross-Entropy Loss as my Loss ...
0
votes
0answers
5 views

Strange situation with softmax activation vs sigmoid activation in the last layer of a binary classification problem?

I am building a neural network as a binary classifier with one output neuron at the last output layer. I deliberately balancing out my train label so that the number label corresponding to the binary ...
0
votes
0answers
21 views

Conceptual explanation of taking derivative of sigmoid function in neural network

In a neural network, we have a bunch of inputs and corresponding weights + a bias which are represented by a multivariable equation. Now we squash this whole equation with a sigmoid function. How ...
0
votes
0answers
32 views

Why is learning slower for a sigmoid activation function in a neural network?

Andrew Ng in one of his deep learning course videos says that the sigmoid function acts as a slow learner in a neural network. My intuition is that the sigmoid as an activation function contributes ...
1
vote
1answer
31 views

Statistical comparison of numerous nonlinear model parameters

I have 84 data sets (n=3) corresponding to 28 conditions (sample composition and temperature) and have fit my data set to the following nonlinear model using MATLAB nonlinear curve fitting: $$y = \...
1
vote
0answers
24 views

the loss suddenly was stuck to a very large value after having several epoch

When I changed the final activation layer in the same model from softmax to sigmoid in order to multilabel classification, the loss would suddenly get stuck at a very big value after several epoch. I ...
1
vote
1answer
66 views

Neural Network: Matlab uses different activation functions for different layers - why?

I have trained on matlab an Artificial Neural Network with one input layer, one hidden layer and one output layer (my output is values between zero and one, which I turn into 0 or 1 according to a ...
1
vote
1answer
86 views

Normalizing continuous features using sigmoid function

Can you use the sigmoid function to normalize continuous features that have no theoretical maximum value but tend to cluster around [-1, 1]? Although using the sigmoid function would be a non-linear ...
1
vote
0answers
37 views

Can binary cross entropy loss be used for non-binary data? [duplicate]

I am following this keras tutorial to construct a convolutional MNIST autoencoder. The decoder has a sigmoid activation function and the entire autoencoder is trained with binary_crossentropy loss. ...
0
votes
1answer
43 views

Sigmoid function calculation range

I know that for big datasets we should try to consider calculations effort and try to minimize execution speed if it does not harm quality. In many models, like regression, neural network, probably ...
0
votes
0answers
32 views

How cost function of Logistic Regression is derived? [duplicate]

How the cost function of logistic regression given below is derived ? Here Sigmoid function is used for logistic regression. After heavy search on internet i have come up here to put on my question , ...
0
votes
0answers
32 views

Understanding interpolation results in a sigmoid regression

I have a doubt about the results I got from an interpolation even though it was performed by a statistical software (SigmaPlot). I have the variable X that is the time expressed in hours and the ...
0
votes
0answers
41 views

Calculate LC50 from a non-sigmoid dose response curve

Is it possible to calculate LC50 (concentration that is lethal for 50% of the test organisms), when my dataset does not have a sigmoid dose response curve? For my experiment, I had 6 treatments ...
1
vote
1answer
24 views

Can we use perceptron training algorithm to train a single neuron with sigmoid activation?

The perceptron training algorithm is summarized as: Apply the inputs and calculate the output $ y $ Compare with the desired output yd and calculate error $e = y-y_d$ Update the weights based on the ...
0
votes
0answers
12 views

How is the output bias decided when using sigmoids to approximate functions

In this link : http://neuralnetworksanddeeplearning.com/chap4.html It has been said that, to approximate functions, the output bias has to be (−m+1/)h. Here is a snapshot of that part: I do ...
0
votes
0answers
20 views

Why are only activations used in softmax set to zero and not the weight matrix during initial phase of learning in neural networks?

I am reading Understanding the difficulty of training deep feed-forward neural networks. While the whole paper itself is very tough, there is one section is particularly not making sense to me. Some ...
0
votes
2answers
61 views

How to normalize the output of a neural network [duplicate]

We have a VGG16 network trained from scratch with a Sigmoid output function. We have 6 classes and the usual output looks like this: ...
0
votes
0answers
37 views

How to adjust the activation function to output y values more than 1?

This link introduces a neural network to solve a classification problem. It uses the sigmoid function as the activation function for both of the two layers (one hidden and one output layer). Can I ...
0
votes
2answers
188 views

With Sigmoid activation and Softmax normalization with cross entropy, are we fitting distributions?

Let's consider I have a multi layer neural network that is doing multi class classification. So each input sample belongs to one on N classes. Now, lets say the last layer has Sigmoid activation ...
0
votes
0answers
101 views

Convnet learning: all weights tend to zero and left only with bias

I am working on a conv net that deals with a classification problem and wonder what sort of mistake I am making to get this sort of problem? I have nine outputs - I can't use softmax as the nature of ...
0
votes
1answer
119 views
0
votes
1answer
105 views

Glorot/ Xavier Init: for sigmoid and tanh?

My question is about Xavier Glorot Init. The assumptions that they make are that they approximate the activation function linearly, that this function has f'(0) = 1 and that we set the bias to 0, as ...
4
votes
1answer
208 views

Is there something like softmax but for top k values?

I have a dataset with binary labels of which exactly k outputs are 1, on which I want to train a neural network. If k=1, softmax can do the job of representing the output distribution. I am interested ...
0
votes
0answers
29 views

Why does the scaled probit function coincide with the logistic sigmoid function at lambda = pi/8?

I'm currently working through through "Pattern Recognition and Machine Learning" by Christopher Bishop, and chanced upon this question while working towards my finals: My two questions are: ...
3
votes
2answers
74 views

What is the interpretation of the “C” parameter in the five parameters logistic curve?

I'm using the following equation for the 5-parameters logistic curve: $$ y = A + \frac{D-A}{\Bigl(1+\exp\bigl(B(C-x)\bigr)\Bigr)^S} $$ What is the interpretation of the $C$ parameter? I found some ...
1
vote
1answer
135 views

Generate odds ratios across deciles / quantiles of an indpendent variable

With reference to the following figure from Bellomo et al., 2011: How exactly are the odds ratios across the deciles 'referenced against the 4th decile' calculated? My initial impression is that a ...
2
votes
0answers
87 views

How to aggregate calibration curves which were created in cross validation?

When looking into Scikit's CalibratedClassifierCV I noticed that the object needs to keep multiple calibrated classifiers in memory to average the results in real time. I understand that these ...
0
votes
3answers
130 views

Can the vanishing gradient problem be solved by multiplying the input of tanh with a coefficient?

To my understanding, the vanishing gradient problem occurs when training neural networks when the gradient of each activation function is less than 1 such that when corrections are back-propagated ...
9
votes
3answers
2k views

Logistic function with a slope but no asymptotes?

The logistic function has an output range 0 to 1, and asymptotic slope is zero on both sides. What is an alternative to a logistic function that doesn't flatten out completely at its ends? Whose ...
0
votes
0answers
233 views

How can I even out the output of the sigmoid function?

I'm applying a sigmoid function to an array of values but the output is very concentrated near 1 (i.e. 25th percentile is 0.999). I think this is because the input array takes on values that are quite ...
2
votes
1answer
90 views

Is there a Gaussian Process Kernel that limits functions to sigmoids?

I am modeling a large number of Dose-response curves. I have strong reason to believe that the generating function will be sigmoidal against the concentration of the assay (Michaelis-Menten kinetics). ...
1
vote
1answer
240 views

Counting the number of neural network parameters [duplicate]

I am slightly confused by counting the number of NN parameters. Let's assume there is a NN with 4-dim vector as an input, then comes 5-dim hidden layers, and another one 6-dim hidden layer. There is a ...
0
votes
1answer
498 views

How to convert IRT theta score to a percentage score

I am trying to implement an adaptive test using 3PL IRT model. We need to screen the candidates and label their expertise as ...
2
votes
2answers
2k views

When is logit function preferred over sigmoid?

I found out that logit and sigmoid functions are inverse of one another, and are used in binary classification, but is there a preference of one over another in any circumstances, or can they be used ...
1
vote
0answers
31 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
3
votes
2answers
3k views

Softmax weights initialization

I am a new to deep learning and neural networks, and I need to know if there is a good weights initialization method to use if the activation function is Softmax like Tanh, ReLU and Sigmoid. Related ...
0
votes
1answer
459 views

Compare two sigmoid shape curves

I have x and y-values for two sigmoid curves (probit). x-values are the same for both curves,...
3
votes
1answer
865 views

Does it make sense to use `logit` or `softplus` loss for binary classification problem?

With $z$ is the logit, $p \in \{1, 0\}$ is the class. Usually binary classification problem use sigmoid and cross-entropy to compute loss: $$\mathcal{L_1} = - \sum{p \log \sigma(z) + (1-p) \log (1-\...
4
votes
0answers
242 views

Has Arcsinh ever been considered as a neural network activation function?

The function $y = arcsinh(x)=ln(x+\sqrt{x^2+1})$ has some nice features that I could imagine being useful as an activation function in a neural network. It has sigmoid behaviour around zero, but far ...
2
votes
2answers
498 views

Question about Sigmoid Function in Logistic Regression

This is with reference with Andrew Ng's video on Logistic Regression, I just want to confirm a small doubt I have. I get the basic idea of Logistic Regression that $z=\theta^Tx$ Where $\theta$= ...
2
votes
0answers
107 views

How does Bayes' rule on two exponentials suggest a sigmoid?

In Platt's 1999 paper on turning support vector machine output into a probabilistic score, he says Bayes rule on two exponentials suggests using a parametric form of a sigmoid where he cites this ...
6
votes
1answer
3k views

Finding the slope at different points in a sigmoid curve

This is my data. x <- c(0.5,3.0,22.2,46.0,77.3,97.0,98.9,100.0) plot(x, pch = 19) I want to fit a curve through these points and then calculate the slope at ...
6
votes
3answers
2k views

Why do we use the natural exponential in logistic regression?

I would like to intuitively understand the benefit of using the natural exponential in the sigmoid function used in logistic regression. Why should it have to be $e^x$ instead of, for example $2^x$?
44
votes
5answers
32k views

Why is tanh almost always better than sigmoid as an activation function?

In Andrew Ng's Neural Networks and Deep Learning course on Coursera he says that using $tanh$ is almost always preferable to using $sigmoid$. The reason he gives is that the outputs using $tanh$ ...
1
vote
0answers
124 views

logistic regression - why we forget assumption on data?

I read logistic regression formula i.e. $ \log\frac{P(C_1|X)}{P(C_2|X)}=w^Tx+w_o$ but this equation is true if we have $P(X|C_1)$ and $P(X|C_2)$ sampled from two Gaussian with the same covariance ...
1
vote
1answer
88 views

Difference between logistic regression models for classification problems

In various papers, I had often seen the logistic regression model for classification problems written in two forms. $$p(y =\pm1|\mathbf{x},\mathbf{w}) = \sigma(y\mathbf{w}^{T}\mathbf{x}) = \frac{1}{1+...
4
votes
0answers
1k views

Expected value of softmax transformation of Gaussian random vector

Let $\mathbf w_1,\mathbf w_2,\ldots,\mathbf w_n \in \mathbb R^p$ and $\mathbf v \in \mathbb R^n$ be fixed vectors, and $\mathbf x \sim \mathcal N_p(\boldsymbol{\mu}, \mathbf{\Sigma})$ be an $p$-...
0
votes
1answer
62 views

Finding the center of a logistic curve

Given a sigmoidal/logistic curve p what's the general procedure to finding at what value of x is the curve centered?
1
vote
2answers
6k views

How to model positive S-shaped-function? [closed]

I need to transform my data into a function shown below. My data should fit in the range from 0 to 1. The inflection point should be on 0.5. How I can do this mathematically? Is there any similar ...
0
votes
1answer
418 views

Modification of Sigmoid function

I need to model my data into a function like shown in the following picture. But how I can do this mathematically? Is there any similar function to model data that should increase on smaller values ...