I'm aware of the fact that probability estimates can be biased in logistic regression when dealing with unbalanced classes. When looking at the log-likelihood function ... $$ ℓ(β)= ∑ 𝑦_𝑖 *\log 𝑝(𝑥_i)+(1−𝑦_𝑖)*\log (1−𝑝(𝑥_𝑖)) $$ ... it makes sense to me that the more frequent class has a somewhat higher weight. For instance, if $y_i = 0$ is 9 times more frequent than $y_i = 1$, the part $(1−𝑦_𝑖)*\log (1−𝑝(𝑥_𝑖))$ of the equation gets much more important, hence getting low $p(x_i)$ for observations where $y_i = 0$ yields more to $ℓ(β)$ than high $p(x_i)$ for observations where $y_i = 1$.

In other words, logistic regression tends to overestimate probability estimates of the more frequent class whereas probability estimates of the rarer class tend to be underestimated.

But in what way, if at all, are coefficient estimates biased? Is only the intercept affected?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.