This might be more appropriate for the Data Science stack exchange, but I'm fairly interested in the most 'scientific' or statistically sound approach. This is probably a fairly routine scenario in some areas, but I've never encountered it in my training.

### The Problem

Say we're running a promotional campaign in an attempt to increase sales.

You can target $N$ people with promotional material out of a total population of $M$ people. You have a budget to initially target a sample of $B$ people, which should allow you to work out who the best $N$ people to target are, out of the total $M$ people ($B < N < M$).

We can assume all $M$ people have a loyalty card so we can track their purchases. This also gives us a bunch of features for each of the $M$ people (such as age, address, job, etc).

### My naive First Thoughts

Randomly send out promotional material to $B$ people and then see if they purchased anything in the next month (or so). This gives me a data set of features for each individual, and whether or not they bought something in the month after receiving the promo.

Now, we can construct a model (a binomial GLM for example) which predicts the probability of purchase after having received the promotional material, conditioned on the individual's feature set (age/job/location etc). Let's call this model A.

I then use that model generate predictions on the total population ($M$ people) and send out the promo's to the $N$ people with the highest predicted probabilities.

However

This approach fails to account for the fact that some of those targeted $N$ people, were probably likely to buy something anyway - and we don't want to waste resources targeting them. What's the best way to approach this?

### Second Thoughts

So we don't want to waste resources on the people who are likely to buy things even if they don't receive any promo material.

We build a second model (on all, or a random sample?) on the $M-B$ people who DIDN'T receive the promo material. This gives us the probability that a person buys something conditioned on their individual features AND having not received any promo material. Let's call this model B.

Now, it seems to me, using the 2 models we have a way of identifying who is likely to make a purchase if they did AND if they didn't receive the promo material.

Let's call $\hat{P}_i^A$ the probability that individual $i$ makes a purchase given he was sent promo material (the output from the first model A), and $\hat{P}_i^B$ the probability that individual $i$ makes a purchase if he wasn't sent any promo material (the output from model B).

Could we simply look at $\Delta P_i = \hat{P}_i^A - \hat{P}_i^B$ and send promo material to the $N$ individuals with the greatest $\Delta P$? Less formally, this would be: to send promo material to individuals who would have the biggest uplift in purchase probability given they have received promo material?.

At this point, you may also want to consider some sort of threshold. For example we may care more about an uplift from $P=0.45$ to $P=0.55$ than we care about an uplift from $P=0.1$ to $P=0.4$. So we might filter on customers whose $\hat{P}_i^A$ is greater than a certain amount.

### Questions

Is this approach reasonable? valid? statistically sound? Would you randomly send out the $B$ promo's? Would you stratify by some variable? What about for the second model?

Would there be a more optimal way of sorting out who should and shouldn't receive the promo material if all we care about is maximising the number of items sold?

• This is mostly about information, and how to encode it -- and in second order a matter of statistics. It helps that you have complete information, so you can distinguish between the different outcomes (buyer/non-buyer). I'm not sure, what you need the/a model for -- you enter somewhat shaky territory. You'd like to do two different things: (a) find a feature from which to draw whether you send promotional material or not and (b) find a way to maximize the revenue/sales. Both are not independent, but there are several considerations that are missing... e.g. cost function for promo material. – cherub Mar 26 '18 at 13:44
• Surely you definitely need a model to do (a)? You can use the output from the model to solve problem (b) when you're happy with your cost/utility function for expected sales/revenue? (although that latter bit might be easier said than done) – dcl Mar 26 '18 at 23:07

My answer to the first query:  it does not seem unreasonable.  But, I would suggest that there are two probabilities from the first model that should be included in your final selection protocol. The first probability would be $\hat{P}_i^{A} = \hat{P}_i^{A|pm}$, the probability that individual $i$ makes a purchase given they were sent the promotional material.  The second probability would be $\hat{P}_i^{A|npm}$, the probability (from the first model) that individual $i$ makes a purchase given they were not sent the promotional material.  It seems that $\hat{P}_i^B$ should be compared first to $\hat{P}_i^{A|npm}$, and then after this comparison, $\hat{P}_i^{A|pm}$ should be compared to some criteria along with $\Delta P = \hat{P}_i^{A|pm} - \max\left(\hat{P}_i^{A|npm},\hat{P}_i^B\right)$ reaching some desired level.
• Thanks for the response. In your suggestion, does model A contain observations that were not sent promotional material? Or does $\hat{P}_i^{A|npm}$ simple come from the same model but ignoring the parameter/coefficient for 'sent promotional material' ? – dcl Mar 25 '18 at 23:32
• So Model A is on all $M$ individuals, model B is on $M-B$? (just realized I've used terrible names for the models). Do we really need model B for the comparison? Why not just ignore the parameter associated with the 'sent promo' when making the comparison? – dcl Mar 26 '18 at 0:11
• Here's a slightly different suggestion: let group B be randomly selected, but only 1/2 randomly receive the promo materials; then Model A can be calculated from this group and applied to the $M-B$ data set to predict both probabilities (getting the promo materials or not). Then, you can use the entire data set (ignoring who was selected for the first mailing) to predict the probability of purchasing. I think these might be the better way to calculate the three probabilities I suggested above. – Gregg H Mar 26 '18 at 1:05