Skip to main content
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
edited tags
Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663
edited tags
Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663
Source Link

What is the proper way to estimate the probability (proportion of time) a rare event occurs?

Often, I need to estimate the probability (proportion of time) a rare event occurs. The standard MLE estimate often gives me extreme estimates since the denominator is usually 1, and the numerator is either 0 or 1, giving me either 100% or 0%.

For example, I am trying to estimate the proportion of web referrals as a result of my email campaign for each of my users. Since the events are rare, most of my users usually have only 1 web referral, and they have either 0 email referral or 1 email referral. In such cases, the MLE estimate is quite unreliable.

Are there standard tricks to correct this over-under estimation? Perhaps something like the laplace smoothing? If yes, how should I go about it?