# Can we use perceptron training algorithm to train a single neuron with sigmoid activation?

The perceptron training algorithm is summarized as:

• Apply the inputs and calculate the output $$y$$
• Compare with the desired output yd and calculate error $$e = y-y_d$$
• Update the weights based on the error: $$w_t = w_{t-1} + \eta ex$$

I studied this for perceptron with step activation and found many examples of the training process for that like AND gate and OR gate.

The question now is: can we use this to train a single neuron with sigmoid activation? Or we must use the gradient descent.

I searched many times and found no answer to this.

I think that the samples that should be classified as one of the two classes (output = 0 or 1) will be trained successfully. However, I'm not sure about the samples that lie between the two classes (0 < output < 1). I am not sure about this and I have no proof.

Assume you use a sigmoid neuron with binary cross entropy loss: $$H(y)=y_d\log y+(1-y_d)\log(1-y)$$ where $$y=\sigma(w^Tx)$$. The gradient wrt $$w$$ would be: \begin{align}\frac{\partial H}{\partial w}&=\frac{\partial H}{\partial y}\frac{\partial y}{\partial w}=\left(\frac{y_d}{y}-\frac{1-y_d}{1-y}\right)y(1-y)x\\&=(y_d(1-y)-(1-y_d)y)x\\&=(y_d-y)x\end{align} When we substitute into the gradient descent rule, we have the following update equation: $$w_t=w_{t-1}-\eta {\partial H \over {\partial w}}=w_{t-1}+\eta(y-y_d)x$$