0
$\begingroup$

For the perceptron algorithm, what will happen if I update weight vector for both correct and wrong prediction instead of just for wrong predictions? What will be the plot of number of wrong predictions look like w.r.t. number of passes? The algorithm of perceptron is the one proposed by Rosenblatt (1958) as below:

enter image description here

My question is that what will happen if we remove if condition and execute update for all instances in each pass.

$\endgroup$
  • $\begingroup$ If you look at the cross-entropy loss function, loss is equal to zero when you have the correct prediction. Hence, in this case, error is not propagated backwards, and the weights not updated accordingly. $\endgroup$ – Buomsoo Kim Sep 15 '17 at 23:27
0
$\begingroup$

In the learning algorithm of the perceptron, the weights are not updated after a correct response.

The learning rule says that the weight vector $\mathbf{w}=(w_i,...w_n)$ is updated according to $w_i(t+1)=w_i(t)+(d_j-y_j(t))x_{ji}$ (see wikipedia). So if the output $y_j$ (obtained applying the $j$th input vector $x_j$) is equal to the desired output $d_j$, that is $d_j=y_j(t)$, the rule becomes $w_i(t+1)=w_i(t)+0$, hence there is no change to the weight.

$\endgroup$
  • $\begingroup$ I'm not sure if we are talking about the same algorithm. I added the algorithm I talked about in the question. Could you please have a look at it? Thank you! $\endgroup$ – ZigZagZebra Sep 16 '17 at 15:44
  • $\begingroup$ Oh I see. I have reported a more "modern" and general version of the algorithm. In the original one, inputs $\mathbf{x}$ and outputs $y$ are $\in \{1,-1\}$. I think that if you update after each pass (regardless of correct or wrong output) then the perceptron will not find the solution. This becase while the original algorithm minimize a meaningful loss function $L(w)=\frac{1}{m} \mathop \sum \limits_1^m {\rm{max(0,}}{{\rm{y}}_i}\left\langle {w \cdot {x_i}} \right\rangle {\rm{)}}$, your modified version does not and add random changes to the parameters after each correct response. $\endgroup$ – matteo Sep 17 '17 at 0:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.