1
$\begingroup$

So, basicaly, me and my team are trying to target our marketing campaigns based on logistic regression's coefficients. The idea is to understand the dimensions that increase or decrease the probability of an user click in the campaign's ads. Through a logistic regression, we concluded that we can achieve statistical significance and we can isolate the effects of each dimension of targeting. That being said, our main goal is the impact of the independent variables and their relationship with the dependent variable, not prediction. The problem is that the dependent variable is a rare event. We already have the model and it' has many coefficients that are statistical significant, but my concern is if they can be biased because of the unbalanced data. I have seen many topics that recomend the King and Zeng method : https://www.cambridge.org/core/journals/political-analysis/article/div-classtitlelogistic-regression-in-rare-events-datadiv/1E09F0F36F89DF12A823130FDF0DA462. But also, I have seen answers that argue that since the Logistic Regressions is a probabilistic model, it's not affected by rare event data and it's coefficients are fine.

King, Gary, and Langche Zeng. "Logistic regression in rare events data." Political analysis 9.2 (2001): 137-163.

$\endgroup$

1 Answer 1

1
$\begingroup$

The issue here is not quite with imbalanced data, which logistic regression handles fine, but in the fact that the imbalance means that you will have few observations of the minority class. If you really do have only a $1\%$ chance of observing a minority case, there's a real chance that you won't see any such cases in $100$ or $200$ observations. Consequently, you need to collect many observations in order to observe enough minority cases, or you need to get creative.

Based on the abstract, the paper you linked wants to avoid the considerable expenses that could go along with needing a large sample size to assure enough minority cases wind up in the data, and it gets creative in order to do so.

First, popular statistical procedures, such as logistic regression, can sharply underestimate the probability of rare events.

This could come from the fact that, even if your true prevalence of minority cases is $1$ in $100$, if you sample $500$ total observations, it is plausible that you would just end up with $2$ minority cases, tricking you into thinking the rate is $0.4\%$ instead of $1\%$.

Second, commonly used data collection strategies are grossly inefficient for rare events data. The fear of collecting data with too few events has led to data collections with huge numbers of observations but relatively few...explanatory variables

If you need $100$ minority cases, and there is considerable class imbalance, you will need to sample a lot more than $200$ total cases like you would expect for a situation where there is a balanced class ratio.

$\endgroup$
7
  • $\begingroup$ Dave, thank you so much for the answer! So, basicly you are saying the problem is not the unbalanced data itself, but the sample needed to estimate the model, right? I don't know if I have too feew observations in my mintory class (and if I can estimate this), but the scenerio it' something like this: 1. Majority class: 500.000 observations 2. Minority class: 2500 observations. The ratio is 0.5%. If we assume that I do not have enough observations on my minority class,it's not a good idea to use the coefficients because they are biased due to the understimation in rare event's probabilities ? $\endgroup$
    – Hermit97
    Dec 14, 2021 at 0:26
  • $\begingroup$ @Hermit97 Be careful in referring to “bias”, as that has a specific definition in statistics that the authors do not mention in the abstract. They refer to “efficiency”. $\endgroup$
    – Dave
    Dec 14, 2021 at 3:13
  • $\begingroup$ Well pointed Dave! But in this part, they explain this say this: ''The mean of a binary variable is the relative frequency of events in the data, which, in addition to the number of observations, constitutes the information content of the data set. We show that this often overlooked property of binary variable models has important consequences for rare event data analyses.[...]'' $\endgroup$
    – Hermit97
    Dec 14, 2021 at 3:19
  • $\begingroup$ ''[...]For example, that logit coefficients are biased in small samples (under about 200) is well documented in the statistical literature, but not as widely understood is that in rare events data the biases in probabilities can be substantively meaningful with sample sizes in the thousands and are in a predictable direction: estimated event probabilities are too small'' $\endgroup$
    – Hermit97
    Dec 14, 2021 at 3:21
  • $\begingroup$ That's why i was worried about the coefficients interpretation for decision making purpose $\endgroup$
    – Hermit97
    Dec 14, 2021 at 3:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.