Out of 84 chances in a week, I want about 3 to be TRUE

Question

I am working on an early phase software startup to help other SaaS (software as a service) companies retain new customers. We will be sending out automatic emails designed to look like they are written in a one-off personal fashion. The contents of the email will be tailored to reflect how much time they have invested in the setup of their SaaS accounts.

It's our goal to send about three emails per new customer per week. Our email app sends emails during "waking" hours (8 a.m. to 8 p.m.) and it looks at the email send history for each new customer's account once per hour. So it's looking at each customer's email history 84 times per week (7*12). Upon checking the email history, it decides whether to send an email.

My approach to coding this function is just to use a probability to trigger an email, with some degree of confidence that over a week the total number of sent emails is going to be about three. This will provide a lot of consistency while at the same time still appearing to be "human" in that it is random in its timing.

I've tried calculating this a few times, and my best calculation

 (1-(3/84))^84 

has returned a probability of .04713, but I've been running a simulation for about three weeks with 2500 customer profiles and only a handful of accounts have produced 3 emails in a week. Clearly my calculations are wrong.

It's been a while since I took a stat course and I'm not entirely confident I'm going to come up with the right answer :)

Your thoughts are appreciated.

Answer 1

Create a list with customers - each listed three times. Shuffle the list. Send an email to the first person on the list. Wait an appropriate time (number of hours in the week divided by three times the number of customers). Repeat.

The above guarantees three random emails sent to each customer. Obviously if you don't want to send during certain hours you can limit the number of hours in the week and adjust accordingly.

Answer 2

If you set the probability $p$ of an email going out at any given hour to $\frac{3}{84}$, then on average each user will receive 3 messages weekly. To verify this, note that this problem is a binomial distribution with $n = 84$. Since the expected value of a binomial distribution is $np$ and you'd like the mean number of emails to be 3, $84p = 3$, and $p = 3 / 84$.

As for the degree of confidence this will roughly equal 3, with this approach the distribution of emails users will receive will look like so:

Note that, while 3 is the expected value, only about 22 percent of user-weeks will have three emails. If you find this distribution over number of emails is too dispersed—i.e. the range of potential values is too broad—or you're not willing to accept a 5 percent chance that a user receives 6 emails, then you may wish to consider another approach.

For example, you could sample 3 values without replacement from the integers from 0 to 83, and then simply schedule the three weekly emails accordingly. You could also choose arbitrary probabilities for the number of emails to be sent to each user, and then randomly choose that many hours at which to send emails. E.g. a 0.5 probability of three emails, 0.25 of two, 0.25 of four, or whatever arbitrary distribution you'd like.

From a development perspective this could prove useful down the road if you later create methods to learn how many emails is most likely to be successful for a given user, and when those messages are most likely to be successful. Meaning, if you already have a random scheduler in place, it'll be easier to swap out for a scheduler that uses the results of a learning algorithm. (Assuming you have some metric of success available for each message, which may be a faulty assumption on my part.)

Answer 3

Seems like you want to number all your customers, 1,2,3,...., and then randomly select from this set of numbers, email the customer with the randomly selected number, as long as they haven't received three emails. You'll need to continuously adjust the number of times the program tries to send an email to ensure the number of tries is 3*number of customers.

So, for example, I have 100 customers I want to email 3 times, randomly interspersed through the week. I set up a program to try and send 300 emails, so I divide (7*12)/300 to get the timing of the efforts. At each effort I randomly pick a number between 1 and 100, and send an email to them, checking to ensure the haven't received three. Near the end of the week the only people receiving emails will be the one or two dozen customers who have only received one or two emails.

In, fact, you could probably setup checks so that everyone receives 1 before you start randomly sending second emails, and so on.

Hope this makes sense.

Edit: SeanEaster's suggestion: "For example, you could sample 3 values without replacement from the integers from 0 to 83, and then simply schedule the three weekly emails accordingly." is somewhat similar to my solution and probably the best way to go. The true probability method could occasionally have a customer receive 5 or more emails, small chance but there - it's gonna happen at some point - and that seems like way too much. Like it would drive them away.

Edit: Bad pseudocode for what I'm suggesting (not great at pseudocoding):

I'm assuming every Saturday or Sunday, you adjust the No. of customers to reflect the latest figures.

While (execute=true){

  if(Sunday, 7 am)   
  {
    Reset everything
    start timer   
  }

  if(timer = 84/(No. of customers * No of emails desired))   {  

    reset timer    
    emailSent = false

    do{

      randomly select from 1 - No. of customers

      if(No. of emails sent to customer < 3){
        send email to customer
        emailSent = true
      }

    }
    while(emailSent = false)   
  }  

}

I think that shows how the do-while loop would ensure that emails are only sent out if the customer has received less than 3 (each customer will never receive more than three) and the timer functionality ensures that the correct number of efforts is made for everyone to receive three, with the efforts spaced out evenly throughout the week. I can't think of how I would continuously update the number of customers - never should have said that. Any code i would write would need to be stopped, input new numbers, and then rerun. I still think SeanEaster's suggestion I highlighted before is best - it ensures three emails, randomly spaced across 84 hours, for whatever umber of customers. Obviously, that setup would require adjusting the customer information on some regular schedule as well.