I'm confused on how to plot decision boundary for classifiers.

For example, i'm working with perceptron. So, the formula for decision boundary(if I understand this correctly) is

W1x + W2y + W_bias = 0

It's equal 0 because (again, if i understand this right): the activation function is +1 if the dot product of W and x >0 and -1 if otherwise. This makes the decision boundary equals 0. Is this right?

While this is simple for perceptron, what is the formula for decision boundary logistic regression? It can't be

sigmoid(W1x) + sigmoid(W2x) + W3 = 0

can it?

How do I determine decision boundary formula for logistic regression or any other classifier (particularly nonlinear ones)?


2 Answers 2


For logistic regression, it's actually \begin{gather} p(x) = \frac{1}{1 + \exp(- (\sum_j W_j x_j + b) )} = \frac12 \\ 1 + \exp(- (\sum_j W_j x_j + b) ) = 2 \\ \exp(- (\sum_j W_j x_j + b) ) = 1 \\ - (\sum_j W_j x_j + b) = 0 \\ \sum_j W_j x_j + b = 0 ,\end{gather} exactly the same as for the perceptron.

In general, you can try to solve for $p(x) = \frac12$ when you get a probability estimate (and the prior is bounded), or more generally if your prediction is $\mathrm{sign}(f(x))$, for $f(x) = 0$. But for nonlinear classifiers, this may be difficult to solve, and thus it's often easier to just plot the output of the classifier on a fine grid as Nick suggested.


This is, to me, a matter of taste. In the past what I've done (and what I'd recommend) is to grid your space into a fine grid and run the points on the grid through your classifier. You can then color those points with the given class and plotting those colored points will give you a visual decision bound if your grid is fine enough. Tibshirani and Hastie have some code in their online ESL book for doing that


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.