In many machine learning algorithms such as SVM, GBM, Logistic Regression, etc., are Decision Function and Separating Hyperplane the same?
$\begingroup$
$\endgroup$
asked Aug 13, 2015 at 2:59
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Not quite.
The decision function is a function $f$ which, given an input, returns some measure of confidence in the classification. Typically the predicted class for an input $x$ is $\mathrm{sign}(f(x))$.
The separating hyperplane, or more generally the decision boundary, is the "line" between positive and negative predictions, i.e. the set $\{ x : f(x) = 0 \}$.
$\begingroup$
$\endgroup$
They are similar in that all these models rely on some kind of binning of samples based on various notions of data separation. SVM and kernel methods are a vast generalization of hyperplane separation which allows for separating curves (equivalently separating hyperplanes in transformed coordinates). Classification via Hyperplane separation is arguably the simplest form of classification and is easily implemented by a perceptron. On it's own hyperplane separation just tries to separate two defined regions of space via a hyperplane. So there's no classification going on there.
Decision trees, especially something like random forests bin by "hypercubing," rather then hyperplanes. A typical decision tree will cordone off samples via comparisions of each coordinate, whether it's bigger or smaller then some value. This will result in classification regions that are unions and differences of hypercubes. See this link for example.
