Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
For perceptron algorithm, the output and target values are either $0$ or $1$.
Assume output is $y$ and target is $d$.
From http://lcn.epfl.ch/tutorial/english/perceptron/html/learning.html, we can see that the learning / adjustment of the weights are like$$w_j(t+1)=w_j(t)+\eta(d-y)x$$
But if $d$ and $y$ are either $0$ or $1$, then $d-y$ would be either $-1$, $0$ or $1$, then it seems the learning becomes dependent on the learning rate?
True. The learning rate is the size of the steps to move in a direction. the difference between d and y show's the direction to move and Learning rate says how much to move in that direction. Like a Gradient descent.
Learning is dependent on the learning rate.
Imagine you want to drive to some place. The (d-y) is the direction you want to follow, while the learning rate is the speed with which you are going towards it.
In the case of the single-layered perceptron that you asked about (which is described in some more detail in wikipedia), the learning doesn't depend on the learning rate.
See here for a short explanation, and here for a longer explanation (including a partial proof and code example).
