For 1000 newsletter recipients, I crudely estimate the likelihood (p
) of them reading the next email sent as:
p = number_emails_read / number_emails_received
I also calculate the standard deviation for each recipient. But there's a problem..
The problem
Anyone who received a small number of emails, say 2 emails, and read both, has 100% estimated likelihood to read the next one, and standard deviation of 0 (in other words, the standard deviation tells us the estimate of 100% likelihood is extremely accurate).
But in reality, their likelihood may actually be much lower than 100%, say 50%, and they just happened to read the two sent to them, but may not be anywhere close to 100% likely to read the next email.
In such cases, the very small sample size may lead us to a false positive, and given the inconvenience of receiving junk emails, I want to create a bias against such cases, to reduce false positives (probably at the expense of false negatives, but that is acceptable, if not desirable here)
Question
How can we adjust against this, that is, to penalise the small sample sizes so that we avoid false positives?
What I know so far
A very crude solution could be to simply remove all cases where the sample size is less than a certain value (e.g. <10), so as to avoid the highest risks of false positives.
But I hope there is a more sensible / statistically valid solution