# Theory: Logistic Regression MLE does not exist when data are perfectly separable? [duplicate]

I realize that this is a relatively well published/understood concept, but I'm unfamiliar with how the conclusion is derived. Could someone ELI5 why when the data are perfectly separable, the MLE for a logistic regression $\hat{\beta} = \infty$?

## marked as duplicate by SmallChess, Michael Chernick, mdewey, kjetil b halvorsen, Peter Flom - Reinstate Monica♦ regression StackExchange.ready(function() { if (StackExchange.options.isMobile) return; $('.dupe-hammer-message-hover:not(.hover-bound)').each(function() { var$hover = $(this).addClass('hover-bound'),$msg = $hover.siblings('.dupe-hammer-message');$hover.hover( function() { $hover.showInfoMessage('', { messageElement:$msg.clone().show(), transient: false, position: { my: 'bottom left', at: 'top center', offsetTop: -7 }, dismissable: false, relativeToBody: true }); }, function() { StackExchange.helpers.removeMessages(); } ); }); }); Apr 12 '17 at 13:26

Imagine you have two data points:

1. $x_1 = 1$, $y_1= 1$
2. $x_2 = -1$, $y_2 = 0$

And let's say we're estimating $b$ in $P(Y= 1 \mid X = x) = \frac{e^{bx}}{1 + e^{bx}}$

How does $b=5$ do? Let $P_1$ and $P_2$ be the probability of correctly forecasting the 1st and 2nd point respectively. We have:

• $P_2 = P_1 = \frac{e^5}{1+e^5} = .9933$

Can we do better? Sure. How about $b=6$

• $P_2 = P_1 = \frac{e^6}{1+e^6} = .9975$

With perfect separation, you always have a higher likelihood by increasing $b$ so the maximum likelihood estimate (MLE) doesn't exist. In a sense, the MLE is $b = \infty$. See the nifty graphic on this answer: https://stats.stackexchange.com/a/224864/97925