2
$\begingroup$

Recall the perceptron algorithm:

cycle through all points until convergence

$\text{if }\, y^{(t)} \neq \theta^{T}x^{(t)} + \theta_0\,\{\\ \quad \theta^{(k+1)} = \theta^{k} + y^{(t)}x^{(t)}\\ \}$

Here's a new update equation with $\eta_k$ as the step-size (aka learning rate):

$\theta^{(k+1)} = \theta^{k} + \eta_k y^{(t)}x^{(t)}\\$

The algorithm, at every step, selects a $\theta$ that minimizes the quantity:

$\frac{\lambda}{2}||\theta-\theta_k||^2+Loss_h(y^{(t)}x^{(t)}\theta)$

where

$Loss(y^{(k)} \theta^{(k)} \cdot x^{(k)}) = max\{0, 1-y^{(k)} \theta^{(k)} \cdot x^{(k)}\}$

The closed form of the equation for the perceptron update is:

$\eta_k = \min\{\frac{1}{\lambda},\frac{ Loss (y^{(k)} \theta^{(k)} \cdot x^{(k)} ) }{\left \| x^{(k)} \right \|^2}\}$

How is this derived? I can't find any on this topic. Thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.