I am preparing a presentation on logistic regression. I applied logit model to a data set and now want to check whether my model meets logistic regression assumptions. I don't exactly know how to do so. Any advice on how or online source is welcomed
$\begingroup$
$\endgroup$
4
-
$\begingroup$ statisticssolutions.com/assumptions-of-logistic-regression $\endgroup$– Maxim.KCommented Nov 18, 2014 at 13:31
-
3$\begingroup$ @Maxim.K - It may seem like a good intro, but there is some very bad advice on that page. A quick look suggests ats.ucla.edu/stat/stata/webbooks/logistic/chapter3/… as an alternative. $\endgroup$– rolando2Commented Dec 23, 2014 at 2:52
-
$\begingroup$ Thanks @RichardHardy -- updated url is stats.idre.ucla.edu/stata/webbooks/logistic/chapter3/… $\endgroup$– rolando2Commented Mar 11, 2021 at 14:03
-
1$\begingroup$ @rolando2, here is what I mean by full reference: UCLA IDRE LESSON 3 LOGISTIC REGRESSION DIAGNOSTICS. (Could be fuller, but will do.) $\endgroup$– Richard HardyCommented Mar 11, 2021 at 14:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
In my opinion there are two kinds of logistic regressions : statistical logistic regression and 'machine learning' logistic regression.
Originally, a logistic regression is used to model a binary response variable $y\in\{-1,1\}$ depending on input variable $X = (x_{1},...,x_{n})$ with a Bernouilli distribution : $P(y_{i}=1;X_{i}) = \frac{1}{1+e^{\beta . X_{i}}}$ $i.e:P(y_{i};X_{i}) \frac{1}{1+e^{-y_{i} \beta . X_{i}}}$ Therefore as you need to get an estimation of the parameter $\beta$, you use the maximum likelihood estimation on a dataset (observations): $\max \prod \frac{1}{1+e^{-y_{i} \beta . X_{i}}}$ which requires that your observations are independant and identically distributed.
However this maximisation problem is equivalent to : $\min \sum \log(1+e^{-y_{i} \beta . X_{i}})$ which can be seen as a loss function we need to minimize in order to get a good accuracy on the whole dataset. In this case, no i.i.d assumptions are needed actually. It is just an optimisation problem.
But in both cases, your binary discrimination problem must not be separable to be able to solve the maximisation problem. (If your problem is separable, then one or some of your coefficients will go to infinity as you will be able to have null loss function i.e : $\forall i, \ e^{-y_{i}\beta X_{i}} = 0$.)
-
$\begingroup$ I think $\max \prod \frac{1}{1+e^{-y_{i} \beta X_{i}}}$ is equivalent to $\max \sum \log(1+e^{-y_{i} \beta X_{i}})$, not $\min \sum \log(1+e^{-y_{i} \beta X_{i}})$, so you need a minus sign in front. $\endgroup$ Commented Mar 24, 2020 at 10:52