3
$\begingroup$

Is Logistic regression , Linear regression , SVM a learning algorithm or a model. I See in some literature they say K-NN is a learning algorithm and not a model.

$\endgroup$

2 Answers 2

1
$\begingroup$

A model uses an algorithm for training and decisions. So, the algorithm describes the internal mechanics of how the model decides and is trained. Model changes with hyper-parameter and data, but the algorithm is invariant. For example, Training the KNN classifier, using K=1 or K=5 creates two different models. But, they're using the same algorithm.

$\endgroup$
1
  • $\begingroup$ According to you, is it correct to say that a model is the result of an algorithm ? $\endgroup$ Mar 3, 2020 at 12:48
0
$\begingroup$

Machine learning models are closely associated with their learning algorithms.

The SVM algorithm uses the training data to find a hyperplane that optimizes the separation between positive and negative classes. The SVM model is then a well-defined computation to assign pos/neg labels to new instances. Same for Linear Regression models using Gradient Descent Algorithm or Decision Trees using ID3 algorithm.

If you deliberately excluded K-NN from the rest: lazy learning does not construct a model that generalizes over the entire training distribution and instead waits until it is given new instances to classify. Since learning is postponed and there is no fixed model, 'learning algorithm' might be preferred.

In machine learning, the terms hypothesis and model and algorithm are often used interchangeably. The terminology to describe these concepts is not standardized. I think the following (I think from "The Elements of Statistical Learning"), is widely accepted:

  • The phrase single hypothesis refers to a single probability distribution or function. An example is the polynomial 2x^2 + 3x + 1.
  • The word model refers to a set of probability distributions or family of functions with the same functional form. An example is the set of all quadratic functions.

As a generic term, hypothesis refers to both single hypotheses and models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.