2
$\begingroup$

I am preparing a presentation on logistic regression. I applied logit model to a data set and now want to check whether my model meets logistic regression assumptions. I don't exactly know how to do so. Any advice on how or online source is welcomed

$\endgroup$
4

1 Answer 1

-2
$\begingroup$

In my opinion there are two kinds of logistic regressions : statistical logistic regression and 'machine learning' logistic regression.

Originally, a logistic regression is used to model a binary response variable $y\in\{-1,1\}$ depending on input variable $X = (x_{1},...,x_{n})$ with a Bernouilli distribution : $P(y_{i}=1;X_{i}) = \frac{1}{1+e^{\beta . X_{i}}}$ $i.e:P(y_{i};X_{i}) \frac{1}{1+e^{-y_{i} \beta . X_{i}}}$ Therefore as you need to get an estimation of the parameter $\beta$, you use the maximum likelihood estimation on a dataset (observations): $\max \prod \frac{1}{1+e^{-y_{i} \beta . X_{i}}}$ which requires that your observations are independant and identically distributed.

However this maximisation problem is equivalent to : $\min \sum \log(1+e^{-y_{i} \beta . X_{i}})$ which can be seen as a loss function we need to minimize in order to get a good accuracy on the whole dataset. In this case, no i.i.d assumptions are needed actually. It is just an optimisation problem.

But in both cases, your binary discrimination problem must not be separable to be able to solve the maximisation problem. (If your problem is separable, then one or some of your coefficients will go to infinity as you will be able to have null loss function i.e : $\forall i, \ e^{-y_{i}\beta X_{i}} = 0$.)

$\endgroup$
1
  • $\begingroup$ I think $\max \prod \frac{1}{1+e^{-y_{i} \beta X_{i}}}$ is equivalent to $\max \sum \log(1+e^{-y_{i} \beta X_{i}})$, not $\min \sum \log(1+e^{-y_{i} \beta X_{i}})$, so you need a minus sign in front. $\endgroup$ Mar 24, 2020 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.