0
$\begingroup$

I have a dataset with information of different individuals and I want to create a model to predict if the individual will take some action or if he won't (1 or 0). E.g.: the individual will buy a product (event = 1) or if he won't (event = 0). Normally, I would use a decision tree or a logistic regression to predict if the event will happen. However, I'm not sure if this is the case for this dataset, since the number of times the event happened in my historical data is very rare (less than 1% of the time). How can I model this?

$\endgroup$
5
  • 1
    $\begingroup$ How many actual ones do you have in the data? $\endgroup$
    – dimitriy
    Commented Jun 5, 2018 at 17:08
  • $\begingroup$ @DimitriyV.Masterov I have 7,000 ones and 1,000,000 zeros $\endgroup$
    – trder
    Commented Jun 5, 2018 at 17:15
  • 1
    $\begingroup$ That sounds like enough data for MLE to work reasonably well, but predictions might still not be all that great. Search this site for "firth logit" or "rare events logit". $\endgroup$
    – dimitriy
    Commented Jun 5, 2018 at 17:39
  • $\begingroup$ How about the method from extreme value theory, would it helps? There will be some distributions regarding extreme events $\endgroup$
    – son520804
    Commented Jun 5, 2018 at 18:11
  • 1
    $\begingroup$ I wrote an answer, but then I thought, it is too general. So I deleted it. This question needs to be less broad. "How can I model this?" with only information that there is a small amount of data, is not a very specific question. $\endgroup$ Commented Apr 4, 2023 at 13:00

1 Answer 1

1
$\begingroup$

With seven-thousand instances of the rare event, it is, at least qualitatively, clear that you are not lacking for rare events. You simply have many more common events than rare events because, well, that is what "rare" and "common" mean.

As has been argued, the major problem with imbalanced data like you have is not so much the imbalance as much as the imbalance leading to few observations of rare events. You, however, have thousands of observations of the rare event, rather than having thousands of total observations yet only a few-dozen observations of the rare event. Consequently, the usual methods are likely to be fine.

Imbalanced problems like this often appear to pose problems, because they often result in few, if any, of the rare events being caught. Much of this comes from using a predicted probability of $0.5$ as a threshold for making hard classifications based on the continuous predictions. Because of how low the prior probability of your rare event is, it is only natural that, unless something in the data is screaming out about a rare event,$^{\dagger}$ the posterior probability (your model prediction) will be low. In this case, you might consider the ratio of the predicted to the prior probability. If the rare event happens with a probability of just $7000/(1000000+7000)=0.006951341$, yet you predict a probability of $0.1$, that prediction, despite favoring the common event ($90\%$ chance of the common event), is more than fourteen times higher than the usual probability of a rare event.

$^{\dagger}$This can happen. Have you ever seen/heard/smelled something highly unexpected yet quickly known what it was?

$\endgroup$
1
  • $\begingroup$ Class imbalance is such a common occurrence with so many misconceptions that there is a Meta post about it that is full of links to lead you down the rabbit hole. $\endgroup$
    – Dave
    Commented Apr 4, 2023 at 12:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.