1
$\begingroup$

I'm reading Hands-On Machine Learning and the author states that:

You may have noticed the fact that the Perceptron learning algorithm strongly resembles Stochastic Gradient Descent. In fact, Scikit-Learn’s Perceptron class is equivalent to using an SGDClassifier with the following hyperparameters: loss="perceptron", learning_rate="constant", eta0=1 (the learning rate), and penalty=None (no regularization).

Can someone please explain why?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Maybe tell us what you already know and what is unclear for you? $\endgroup$
    – Tim ♦
    Apr 24, 2021 at 13:46

1 Answer 1

Reset to default
1
$\begingroup$

Both perceptron learning and SGD involve partial derivatives of values of an objective function (SGD) or prediction-error (perceptron) with respect to coefficients. Both problems are essentially solved iteratively via an updating scheme. Both cases are also function minimization procedures. Independent of perceptron learning, derivates don't have to be employed, since one can use finite differencing, or e.g., the Gauss-Siedel method.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ What you say here is also true of batch gradient descent. I don't think you've touched on the crucial similarity, though. Everything is the same, except that perceptron uses error-driven updates and traditionally assumes a learning rate of 1. $\endgroup$
    – Arya McCarthy
    Apr 24, 2021 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.