Logistic regression on 2 concentric circles

Question

If we have 2 concentric circles of different radius and points on the same circle corresponds to the same data class then can model this problem to be solved by logistic regression. I don't think we can since the data points are not linearly separable.

Answer 1

In practice, yes, it can.

Let's say the circles are of radius $\frac{1}{\sqrt{2}}$ and $\sqrt{\frac{3}{2}}$, just to make things easier to follow (*).

Then the following functional form works when thresholded to classify the points:

$$ P(y | x_1, x_2) = \frac{1}{1 + \text{exp}(x_1^2 + x_2^2 - 1)} $$

On the circle of radius one half:

$$ P(y | x_1, x_2) = \frac{1}{1 + \text{exp}(x_1^2 + x_2^2 - 1)} = \frac{1}{1 + \text{exp}(-0.5)} > 0.5$$

On the circle of radius three halves

$$ P(y | x_1, x_2) = \frac{1}{1 + \text{exp}(x_1^2 + x_2^2 - 1)} = \frac{1}{1 + \text{exp}(0.5)} < 0.5$$

So if you threshold the probabilistic predictions at $0.5$, you'll classify correctly.

Note that you do need to add some derived features, in this case the squares of the coordinates. This is very common in practice to deal with these kinds of situations.

(*) Note that this isn't really a restriction, you can always scale your coordinates in a way to map any two concentric circles to this specific situation.