I'm a newbie to Machine Learning and I'm not very good at math. I have read Learning From Data - A Short Course and met this Exercise 4.6 on page 133:

We have seen both the hard-order constraint and the soft-order constraint. Which do you expect to be more useful for binary classification using the perceptron model? [Hint: $ sign(w^{T}x) = \> sign(\alpha w^{T}x) $ for any $ \alpha > 0 $.]

From the hint of the question, I have guessed the answer: Soft-order constraint does not actually "regularize" the perceptron model at all (I'm not sure if I used the right word here, what I mean is that soft-order constraint does not "reduce" the "complexity" of the hypothesis set of the perceptron model), and so the hard-order constraint is more useful for binary classification using the perceptron model.

However, if the above answer is correct, then I do not understand how regularized logistic regression regularize perceptron hypothesis set in the above binary classification task?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.