2
$\begingroup$

We are given a training dataset consisting of a list of feature vectors $x_1, \ldots, x_N \in \mathbb R^d$ and corresponding labels $y_1, \ldots, y_N \in \{-1, 1\}$. Let $\hat x_i$ be the augmented feature vector obtained by prepending a $1$ to the vector $x_i$. We seek to find a vector $\beta \in \mathbb R^{d+1}$ such that if $y_i = 1$ then $\hat x_i^T \beta > 0$, and if $y_i = -1$ then $\hat x_i^T \beta < 0$. We choose $\beta$ to be a minimizer of the function $$ L(\beta) = \sum_{I=1}^N e^{-y_i \hat x_i^T \beta}. $$ What is the name of this particular binary classification algorithm, where $\beta$ is chosen to minimize this cost function $L$? Is this a popular algorithm?

$\endgroup$
2
  • 2
    $\begingroup$ I'd just call it a linear classifier that minimizes the exponential loss. No name AFAIK, since it's not a common thing to do. One common use of the exponential loss is Adaboost. It's not particularly popular anymore compared to other forms of boosting, but is still regarded as an important historical development. In any case, what you're proposing is not equivalent to Adaboost, even though it uses the same loss function. $\endgroup$
    – user20160
    Mar 6, 2020 at 23:32
  • $\begingroup$ Correct, it's just linear regression, with a log-linear transform. It's just function minimization on the log scale. $\endgroup$
    – user32398
    Mar 7, 2020 at 3:10

2 Answers 2

2
$\begingroup$

I am not sure about the name of the algorithm, but I have encountered it in the context of boosting before. Following resources may help you.

http://cs229.stanford.edu/extra-notes/loss-functions.pdf (page 3)

http://cs229.stanford.edu/extra-notes/boosting.pdf (page 3)

$\endgroup$
2
  • $\begingroup$ Those notes are great, thanks. (They mention the algorithm in my question, but they don't give it a name.) $\endgroup$
    – littleO
    Mar 6, 2020 at 22:56
  • $\begingroup$ Based on the notes you shared and comments above, I think this algorithm has no special name but could be called a "linear classifier using the exponential loss function." So I'm going to accept your answer. $\endgroup$
    – littleO
    Mar 7, 2020 at 9:42
-1
$\begingroup$

Support Vector Machine. This algorithm want to find a best hyper-plane (minimized the distance between categories ) to separate different categories.

$\endgroup$
1
  • 1
    $\begingroup$ SVM uses a different loss function, though. SVM is only one of many possible linear classifiers. $\endgroup$
    – littleO
    Mar 6, 2020 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.