Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am working on an early phase software startup to help other SaaS (software as a service) companies retain new customers. We will be sending out automatic emails designed to look like they are written in a one-off personal fashion. The contents of the email will be tailored to reflect how much time they have invested in the setup of their SaaS accounts.

It's our goal to send about three emails per new customer per week. Our email app sends emails during "waking" hours (8 a.m. to 8 p.m.) and it looks at the email send history for each new customer's account once per hour. So it's looking at each customer's email history 84 times per week (7*12). Upon checking the email history, it decides whether to send an email.

My approach to coding this function is just to use a probability to trigger an email, with some degree of confidence that over a week the total number of sent emails is going to be about three. This will provide a lot of consistency while at the same time still appearing to be "human" in that it is random in its timing.

I've tried calculating this a few times, and my best calculation

 (1-(3/84))^84 

has returned a probability of .04713, but I've been running a simulation for about three weeks with 2500 customer profiles and only a handful of accounts have produced 3 emails in a week. Clearly my calculations are wrong.

It's been a while since I took a stat course and I'm not entirely confident I'm going to come up with the right answer :)

Your thoughts are appreciated.

share|improve this question
7  
Although it's a good statistical question, if I were at the receiving end of this I would likely flag you as a spammer after the end of the first couple of weeks and discontinue any relationship with you. If there's any chance your new customers could perceive these contacts as negatively as I do, you might consider rolling out this program with a small sample of the population first. –  whuber Aug 3 at 17:24
    
I'm not sure the time is something I would ever look at to see if I thought something was being sent by a human. If I were, I would guess I would maybe be tipped off if it were a round number. Have you considered just having the program wait a random number of minutes after it makes the relevant determination before firing off the e-mail? Alternately, don't do it at all, per @whuber. I'd be furious if someone sent me 3 e-mails a week about setting up my profile. –  Paul Aug 4 at 2:36
2  
Hey @paul I totally agree in principle. I should say that I have abstracted this question; we're actually doing an action on a time interval that is not sending emails; I "changed the names to protect the innocent" because the actual action is proprietary and is confidential. For the record I think spam is ruining email. –  andrewniesen Aug 4 at 12:47

3 Answers 3

up vote 3 down vote accepted

Create a list with customers - each listed three times. Shuffle the list. Send an email to the first person on the list. Wait an appropriate time (number of hours in the week divided by three times the number of customers). Repeat.

The above guarantees three random emails sent to each customer. Obviously if you don't want to send during certain hours you can limit the number of hours in the week and adjust accordingly.

share|improve this answer
    
I believe this is the right approach, given large variance in number of emails sent per customer due to the small sample set, as explained by @seaneaster. For those interested in how this will be programmed, I will build an array using PHP, insert 3 instances of each user's email address into the array. Then I will randomly shuffle the array, insert the resulting values in the randomized order into a database table, and use that as a queue. With a sending block of 2500 customers, we will pull about 89 customers per hour to send emails to, pulling from the top of the queue. 2500*3/84 = ~89 –  andrewniesen Aug 4 at 12:58
    
@andrewniesen Have you planned how to add newly added users to the queue? –  SeanEaster Aug 4 at 13:19
    
You can add new names in proportion to the fraction of week left (3 for the first 17%, 2 for the next 33, 1 for the next 33, and one for the last 17), the shuffle again and keep going. This will keep things "about as good as it can be". –  Floris Aug 4 at 14:00
    
@floris - I think this is a great approach to keeping the number of emails being sent to individuals spread out evenly over the week period. As per my comment above, this question is abstracted; the randomized events that we are talking about (which aren't actually emails) would be more acceptable in quick-succession than my abstracted/hypothetical email analogy, so my fault for not creating a perfect email analogy. I do like the way you think! –  andrewniesen Aug 4 at 14:31
    
@andrewniesen I am glad I was able to help. Amazing what being stuck at an airport for six hours does to the productivity of others... :-) –  Floris Aug 4 at 14:51

If you set the probability $p$ of an email going out at any given hour to $\frac{3}{84}$, then on average each user will receive 3 messages weekly. To verify this, note that this problem is a binomial distribution with $n = 84$. Since the expected value of a binomial distribution is $np$ and you'd like the mean number of emails to be 3, $84p = 3$, and $p = 3 / 84$.

As for the degree of confidence this will roughly equal 3, with this approach the distribution of emails users will receive will look like so:

Binomial distribution PMF with p = 3/84

Note that, while 3 is the expected value, only about 22 percent of user-weeks will have three emails. If you find this distribution over number of emails is too dispersed—i.e. the range of potential values is too broad—or you're not willing to accept a 5 percent chance that a user receives 6 emails, then you may wish to consider another approach.

For example, you could sample 3 values without replacement from the integers from 0 to 83, and then simply schedule the three weekly emails accordingly. You could also choose arbitrary probabilities for the number of emails to be sent to each user, and then randomly choose that many hours at which to send emails. E.g. a 0.5 probability of three emails, 0.25 of two, 0.25 of four, or whatever arbitrary distribution you'd like.

From a development perspective this could prove useful down the road if you later create methods to learn how many emails is most likely to be successful for a given user, and when those messages are most likely to be successful. Meaning, if you already have a random scheduler in place, it'll be easier to swap out for a scheduler that uses the results of a learning algorithm. (Assuming you have some metric of success available for each message, which may be a faulty assumption on my part.)

share|improve this answer
    
this is exactly right...from my reading and memory of statistics - I just couldn't wrap my mind around it because it's been a while since I took a stat course. And, I was making it more complicated than it needed to be. Thanks for clarifying. I think that if I were dealing with a larger sample set (number of times we would consider sending an email in a week to far more than 84) we would more reliably be able to get ~3 emails sent per user, right? In other words, the normal distribution that you've drawn would be "skinnier" with a smaller standard deviation, right? –  andrewniesen Aug 4 at 12:48
    
@andrewniesen Not quite. It's true that a distribution with the same mean and lower standard deviation would be skinnier, but that won't be the case with these restrictions. The variance of a binomial distribution is $np(1-p)$. Assuming you keep $p$ equal to $\frac{3}{n}$, which you must to keep the expected number of successes at 3, variance approaches 3 as $n$ grows. To see how this would look, try plotting a Poisson distribution with $\lambda = 3$. (The Poisson can be thought of as the limit of the binomial as $n$ grows, $p$ shrinks, and $np$ remains constant.) –  SeanEaster Aug 4 at 13:31

Seems like you want to number all your customers, 1,2,3,...., and then randomly select from this set of numbers, email the customer with the randomly selected number, as long as they haven't received three emails. You'll need to continuously adjust the number of times the program tries to send an email to ensure the number of tries is 3*number of customers.

So, for example, I have 100 customers I want to email 3 times, randomly interspersed through the week. I set up a program to try and send 300 emails, so I divide (7*12)/300 to get the timing of the efforts. At each effort I randomly pick a number between 1 and 100, and send an email to them, checking to ensure the haven't received three. Near the end of the week the only people receiving emails will be the one or two dozen customers who have only received one or two emails.

In, fact, you could probably setup checks so that everyone receives 1 before you start randomly sending second emails, and so on.

Hope this makes sense.

Edit: SeanEaster's suggestion: "For example, you could sample 3 values without replacement from the integers from 0 to 83, and then simply schedule the three weekly emails accordingly." is somewhat similar to my solution and probably the best way to go. The true probability method could occasionally have a customer receive 5 or more emails, small chance but there - it's gonna happen at some point - and that seems like way too much. Like it would drive them away.

Edit: Bad pseudocode for what I'm suggesting (not great at pseudocoding):

I'm assuming every Saturday or Sunday, you adjust the No. of customers to reflect the latest figures.

While (execute=true){

  if(Sunday, 7 am)   
  {
    Reset everything
    start timer   
  }

  if(timer = 84/(No. of customers * No of emails desired))   {  

    reset timer    
    emailSent = false

    do{

      randomly select from 1 - No. of customers

      if(No. of emails sent to customer < 3){
        send email to customer
        emailSent = true
      }

    }
    while(emailSent = false)   
  }  

}

I think that shows how the do-while loop would ensure that emails are only sent out if the customer has received less than 3 (each customer will never receive more than three) and the timer functionality ensures that the correct number of efforts is made for everyone to receive three, with the efforts spaced out evenly throughout the week. I can't think of how I would continuously update the number of customers - never should have said that. Any code i would write would need to be stopped, input new numbers, and then rerun. I still think SeanEaster's suggestion I highlighted before is best - it ensures three emails, randomly spaced across 84 hours, for whatever umber of customers. Obviously, that setup would require adjusting the customer information on some regular schedule as well.

share|improve this answer
    
@SeanEaster's answer demonstrates why this will not work. Much of the time the program will not send out enough e-mails by the end of the week. Much of the rest of the time it will get them all sent out too soon. –  whuber Aug 3 at 17:19
    
@whuber, I really respect you, but I don't think you understand my solution. My solution suggest programming it to ensure 3 emails per customer are sent, no more no less. If it randomly picks someone who has received 3, it discards that pick and picks another. It does this until it finds someone who has received less than 3; in this manner everyone gets three emails, spread across (No. of customers * 3) times, spaced at (7*12)/(No. of customers) intervals throughout the week. –  traggatmot Aug 3 at 17:24
    
OK, what do you do when at the end of the week only 280 or fewer emails have been sent? (The chance of that happening is 13%, which isn't small.) Note, too, that your procedure is far from random: much of the time the 300 emails will have been sent well before the end of the week. –  whuber Aug 3 at 17:26
    
I feel like I'm missing something, cause you're too smart to not get this. 100 customers, goals of 3 emails. Means 300 chances, spread across 84 hours. So, starting at Sunday at 7 am, every 16.8 minutes, I randomly pick a number from 1-100 and try and send an email to that person. I have a conditional in my program that checks to see if they've been sent three emails. If they haven't, I send them one, if they have, I pick a new number. Suppose I pick 1 three times in a row, no problem, the get sent three emails - the next time I pick 1, I see they've got three emails and pick anew. –  traggatmot Aug 3 at 17:28
1  
@whuber added info to my answer to try and explain further. –  traggatmot Aug 3 at 18:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.