Confused about the basic maths in Incremental/Uplift modelling

Question

I have recently gotten to study Incremental modelling, which is used in Marketing to study the incremental impact of a certain action on a treatment group against a control group.

However, I am very confused about the very basics of this form of modelling: specifically about the formulae used in its calculations. Consider the following example, please:

|CampaignName | Group Type | TotalPeople | Respondents | AverageSpend |
|-------------|------------|-------------|-------------|--------------|
|ToysRus      | Control    | 80          | 10          | 30           |
|ToysRus      | Treatment  | 800         | 130         | 39           |

Supposing that we were to find out:

1. The response rate for the treatment group
2. The incremental response rate
3. The incremental spend per customer
4. Total incremental spend due to targetting

I understand that for 1., we simply use the formula:

No. of respondents in the treatment group/Total no. of people in treatment group

And this will be the rate of response in treatment group - the percentage of people from the treatment group that responded to our 'treatment'. Likewise we can find the rate of response for the control group.

For 2., I am confused. I understand that the formula is:

Response rate from treatment group - response rate from the control group

But I personally used the following formula in my practice:

No. of respondents in the treatment group - no. of respondents in control group / Total no. of people in the treatment group

I know that the answer I get due to this is certainly wrong, but I am unable to convince myself as to why the formula is wrong. It seems intuitive to me on first glance. But I think there is basic Math here at play, which very unfortunately I am unable to see presently. I think perhaps it has to do with qualitative and quantitative differences between the groups? That is, one is a control and the other a treatment group, and both are of different sizes, so it makes less sense to subtract them straight away? Like comparing apples and banana? Even if we do get to apply my formula, I don't see how its interpretation would be possible. Yet, I am unable to convince myself!

For 3., I think the following formula would apply:

Average Spend per customer in the treatment group - average spend per customer in the control group.

For 4., I am unsure, but I think we would use:

The incremental response * the total no. of people in the control group

But how to understand this? Are we extrapolating to the control group with our incremental spend, to project how much the 'untreated' group or the control group would spend if we direct our offers/advertisement/etc. at them?

Lastly, I would be grateful if people answering this question could also suggest further reading for me. Perhaps you feel I need to study the basics of proportion, or percentages, in order to get myself comfortable with such basic calculation in Incremental modelling? Or perhaps you feel that there are certain books in Marketing or Incremental modelling that would help me?

Thanks a lot!

Answer 1

No. of respondents in the treatment group - no. of respondents in control group / Total no. of people in the treatment group

This formula doesn't take into account impact control respondents have on the control total population make doesn't account for the population in the control group fully. Think about having a test population with great response but population less than 1000 vs control with significantly slightly higher response with 1 million population.

Check this link out that has some simple functions broken out to do these sorts of things.

http://excelmasterseries.com/D-_Loads/New_Manuals/Confidence_Intervals_in_Excel.pdf

Average Spend per customer in the treatment group - average spend per customer in the control group.

This would be correct but you still want to use right formulas to account for population sizes of test and control.

The incremental response * the total no. of people in the control group

Multiple ways to look at this one. A total sales number can be used if the population sizes are considered to be similar for test and control. Otherwise same process as above. Your formula actually would be used in more of a hypothetical, "what-if" analysis.

If anyone wants to learn more about this there are some stat MOOC courses out there that would be great intros.

Answer 2

Answer to #4 question "Total incremental spend due to targeting" --

I think it is asking what is the incremental for the test group comparing giving them a treatment vs no treatment. so it is based on the same group of 800 people in this case.

That means the result should be calculated that spend by the 800 people if they got the treatment, minus the spend by the same 800 people if they didn't get the treatment. In short, the formula is: total number of people in the test group * incremental spend per customer (which you got from question #3).