Suppose I have data points $(x_1^1, x_2^1), (x_1^2, x_2^2), (x_1^3, x_2^3), \ldots$ in $\mathbf{R}^2$ that fall in one of two classes, $y^i=0$ or $y^i=1$. I can find a linear separator for these points using logistic regression by finding $b_0, b_1, b_2$ that maximize $$P\left(y\;|\; (x_1, x_2)\right) = \frac{1}{1+e^{-(b_0 + b_1x_1 + b_2x_2)}}$$
based on the data. We can do this by gradient ascent, for example. The line that separates the points is given by the set of points $(x_1', x_2')$ that satisfy $$0.5 = \frac{1}{1+e^{-(b_0 + b_1x_1' + b_2x_2')}},$$ which yields $$b_0 + b_1x_1' + b_2x_2' = 0.$$
Right?
Another way to look at this problem is as a regression problem: We can find parameters $\beta_0, \beta_1, \beta_2$ that minimize $$\sum_d (\beta_0 + \beta_1x_1^d + \beta_2x_2^d-y^d)^2.$$ We can do this using the master equations or gradient ascent... So now the line $y=\beta_0 + \beta_1x_1 + \beta_2x_2$ can be viewed in $3d$ as the line that "best" passes through the data? I think the set of points $(x_1', x_2')$ that satisfy $0.5=\beta_0 + \beta_1x_1' + \beta_2x_2'$ will be the same set of points that were found in the first method above, but I'm having trouble seeing why. Thinking about the data as two big clusters in $3d$ seems to be throwing me off. Can someone shed some light on this?
