7
The existing answers aren't wrong, but I think the explanation could be a little more intuitive. There are three key ideas here.
1. Asymptotic Predictions
In logistic regression we use a linear model to predict $\mu$,
the log-odds that $y=1$
$$
\mu = \beta X
$$
We then use the logistic/inverse logit function to convert this into a probability
$$
P(y=1) = \...
4
The asymptotic nature refers to the logistic curve itself. The optimizer, if not regularized, will enlarge the weights of the logistic regression to put $wx$ as far as possible to the left or right per sample to reduce the loss maximally.
Lets assume one feature that provides perfect separation, one can imagine $wx$ getting larger and larger on each ...
3
As a preliminary note, I see that your equations seem to be dealing with the case where we only have a single explanatory variable and a single data point (and no intercept term). I will generalise this to look at the general case where you observe $n$ data points, so that the log-likelihood function is a sum over these $n$ observations. (I will use only ...
2
To look for equivalence one should compare the form of,
$$\hat{\beta} = \underset{\beta}{\text{argmin}} -y\log(\hat{y}) - (1-y)\log(1-\hat{y}) + \lambda||\beta||_2^2,$$
with the posterior distribution whilst keeping a general expression for the prior.
The posterior distribution has form,
$$\pi(\beta|x) \propto \pi(\beta)L(\beta;x).$$
Where $\pi(\beta)$ is ...
2
This has not to do with that specific log loss function.
That loss function is related to binomial/binary regression and not specifically to the logistic regression. With other loss functions you would get the same 'problem'.
So what is the case instead?
Logistic regression is a special case of this binomial/binary regression and it is the logistic link ...
1
I'm not sure if releveling would be appropriate, as some of the coefficients change (for example the intercept)
Changing the reference level will not achieve much when you only have 2 levels of the independent variables. The intercept will change, but it's just a simple re-parameterisation of the model. When you have a lot of levels of a categorical ...
1
Reverse engineering what R package car does for vif of GLM.
The computation is based on the covariance of the parameter estimates. It also uses Generalized VIF which is defined for terms instead of single columns of the design matrix. In the example, every term is one column, so this does not make a difference.
A corresponding Python code for the vif for ...
1
You give the source’s explanation yourself, where it says in your link https://developers.google.com/machine-learning/crash-course/logistic-regression/model-training:
“Imagine that you assign a unique id to each example, and map each id to its own feature. If you don't specify a regularization function, the model will become completely overfit. That's ...
1
Logistic regression is a convex optimization problem (the likelihood function is concave), and it's known to not have a finite solution when it can fully separate the data, so the loss function can only reach its lowest value asymptomatically as the weights tend to ± infinity. This has the effect of tightening decision boundaries around each data point when ...
1
It seems that the outcome is bounded between 0 and 1, so you could use a beta glmm with random intercepts for school districts. GLMMadaptive can fit such a model in R.
Only top voted, non community-wiki answers of a minimum length are eligible
