# Logistic Regression assumption

In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?

• I think you are thinking of SVM, logistic regression has no such assumption. Apr 24, 2020 at 11:25
• medium.com/@akshayc123/logistic-regression-87f7fbb4aaf6 I've read it from here Apr 24, 2020 at 11:31
• That's weird, what they are describing looks very much how SVM works, not logistic regression. Apr 24, 2020 at 11:37

It is actually exactly the opposite: If the data are linearly separable, logistic regression will not converge: the $$\beta$$s will rise infinitely as the logistic function approaches, but never reaches the form of a step function.

Logistic regression minimises the cost function:

$$L(Y|X, \beta) = -\sum_i y_i \ln p_i + (1-y_i) \ln (1-p_i)$$

There is no closed form solution to this and the minimisation has to be performed numerically. Here,

$$p_i = \frac{1}{1+\text{e}^{-\beta x_i}}$$

is the probability of belonging to the class labeled as "1". It is modelled by the logistic function (hence logistic regression), which is bound to $$(0, 1)$$: That means that its logarithm is always negative, going towards $$-\infty$$ as its argument approaches $$0$$. The above cost function would reach its minimum, zero, if the arguments of $$\ln$$, $$p_i$$ and $$(1-p_i)$$, for classes labeled "1" and "0", respectively, could be one. For that to happen, the exponential term in the denominator would need to be either exactly $$0$$ (for the class labeled "1") or $$+\infty$$ (for "0"), and for that to happen, the vector $$\beta$$ would need to have infinite components. Since infinity can never be reached numerically, no numerical algorithm can converge.