Statistical tests for knowing the effect of a campaign on sales

Question

I am currently working on my very first real life Data Science problem and I am facing a bit of a challenge in formulating the solution.

The question is to find out if conducting a campaign has an effect on the sales overall. To do this, we have the data related to customer information, sales invoices, campaigns conducted with reference to the customers.

I thought of the following ways to tackle this:

• We could simply split the customer base into 2 groups, those participating in campaigns and those not, and then just perform a statistical test (not sure which one yet) to see if there is a significant difference in their aggregate sales.
• Instead of comparing the aggregate sales, we could also compare just the frequency/count of sales across the 2 groups (not sure which model to use here)
• We could go for a logistic regression approach where the target variable represents whether a sale was made or not and then check which coefficients had the highest effect on the target

I am not completely sure as to what would be the best approach here. It would be really helpful if I could get some insights into that.

If there are better approaches to tackle this problem, please do share them! I would love to know more about them.

Thanks in advance!

Edit: I would like to add more information regarding the data itself in case it makes a difference in the assumptions made.

I have 3 datasets

• Customers: This table, as expected, contains anonymized information about the customer, what campaign they were subjected to etc.
• Campaign: Information about the campaigns conducted like start and end dates, some descriptions regarding the campaign etc.
• Sales: This contains information about date of sale, the value of purchase, location of sale, etc.

Answer 1

Based on your posts and comments, I imagine your data on two levels. You have a binary outcome, sales (made or not), let’s call it $Y$. As you mentioned customers can have more than one sale. You also have information on customers with no sales. And I think there may be customers who made purchases sometimes but not in others. So, there is a nesting factor which I’ll call $\operatorname{ID}$. You might have predictors at the sales level, denoted as $X$ (maybe the date of sale; some sales could be on holiday season, etc.), and customer level, denoted as $Z$ (e.g., sociodemographic variables). I also assume that participation in a campaign is at the customer level (someone participated or not).

I would start with a random-intercept binary logistic regression model, taking into account this nested structure:

$$ \begin{aligned} \operatorname{Y}_{i} &\sim \operatorname{Binomial}(n = 1, \operatorname{prob}_{\operatorname{y} = 1} = \widehat{P}) \\ \log\left[\frac{\hat{P}}{1 - \hat{P}} \right] &=\begin{aligned} &\alpha_{j[i]} + \beta_{1}{X_1} + \cdots+ \beta_{n}{X_n} \end{aligned} \\ \alpha_{j} &\sim N \left(\gamma_{0}^{\alpha} + \gamma_{1}^{\alpha}Z_1 + \cdots+ \gamma_{n}^{\alpha}{Z_n}, \sigma^2_{\alpha_{j}} \right) \text{, for ID j = 1,} \dots \text{,J} \end{aligned} $$

You can expand this model based on your analytical interests. As I said, I assume that participation in a campaign is at the customer level but that might not be the case, hence the third question raised by PBulls. Some sales could be made before a customer participated in a campaign or some customers could be exposed to campaign longer than others. If that’s the case, the model changes, and you should interpret the results accordingly.

Answer 2

After researching the topic a bit better, I have come up with the following solutions which make the most sense to me.

1. A Chi-Squared Test to compare the count data for the 2 groups (sale, no sale) would be good. Reference here.

2. Basing the conclusion solely on the Chi-Squared test would not be a very strong foundation and hence conducting logistic regression with the variables available would be helpful.

Combining both of these methods should give the result in a good way so that actual decisions can be taken.

If someone else has better answers I would really appreciate it!