1
$\begingroup$

According to me in logistic regression, we just try to get a line (polynomial) and then based on which side the point is from that line the sigmoid gives ans>=0.5 or <0.5, which we then interpret as 1 (if ans>=0.5) and 0 if (ans < 0.5). Here is a simple plot what I mean.

enter image description here

This is just a random line not a line predicted by any logistic regression model. Here we are only having one feature i.e., x. y is the output(0 or 1). The decision boundary here is x = -3 and x = 3.

So when we have 1 feature we try to get a good line to predict the values.

What if we have 2 features we try to get a plane instead?

This is a 3-d plot of a normal sigmoid.

enter image description here

So if we have 2 features say x and y. For these we are trying to get a plane in z axis which separate the points? which is in turn the decision boundary. I am really confused.? Is this what we are doing?

If yes, can someone plot a 3d plot (of sigmoid/decision boundary) for a circle. For eg:

enter image description here

This is also a random plot. Suppose that our points are like this and our model has given us a circle to differentiate between 0 and 1. So can anyone plot a decision boundary of something like this (which is is 3-D).

Improve this question
$\endgroup$
3
  • $\begingroup$ For a 3D analogue of your circle, wouldn’t it just be points of one color inside a sphere and points of another color outside? Such a visualization might prove frustrating, but the idea makes sense, right? I do, however, see that as separate from your question about the sigmoid function, hence my answer not addressing it. $\endgroup$
    – Dave
    Dec 23, 2022 at 6:59
  • $\begingroup$ Assume that on a table the points/marbles are distributes as show in the circle(plot above). now lets assume the red marbles are y=1 and blue one's are 0. Now to make this to 3-D, lift the red points in the air. as table top will be indicating 0 and the red one's lifted in air means 1. So I think cylinder or something like that should be the actual thing in 3-D. sphere can also be a valid here. $\endgroup$
    – varun
    Dec 23, 2022 at 7:18
  • $\begingroup$ Sphere, cylinder, cone, wilting wizard’s hat…all seem plausible in three dimensions, though only a sphere strikes me as analogous to a circle in the plane. // The business about decision boundaries seems rather unrelated to the question about the sigmoid function and perhaps should be posted as a separate question $\endgroup$
    – Dave
    Dec 23, 2022 at 7:26

1 Answer 1

Reset to default
1
$\begingroup$

The sigmoid function takes the linear predictor of the logistic regression and bends it into a range, $[0,1]$, that can interpreted as a probability. The linear predictor can take any real number, which is fine if we think about the log-odds. However, for valid probability values, the must be a transformation to restrict outputs to $[0,1]$.

Any business about a decision boundary is separate from the sigmoid function. In fact, it is totally reasonable to do a decision boundary in terms of log-odds and skip the sigmoid function altogether. Just evaluate if the prediction has positive or negative log-odds, and classify accordingly. This is equivalent to the decision rule in the OP, and it is a worthwhile exercise to consider why.

As always, hard classifications need not be made, since a logistic regression explicitly predicts probability values that can be valuable on their own without considering hard classifications (at least not initially). Our Stephen Kolassa’s answer here gets into some of that.

Improve this answer
$\endgroup$
1
  • $\begingroup$ If we have a circle as a decision boundary (as shown in the post). Does that mean in actual 3rd dimension the plan is a cylinder or something like that? $\endgroup$
    – varun
    Dec 23, 2022 at 7:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.