# Counterfactual prediction with machine learning, sales data

I have a dataset from a supermarket with around 10 thousand products. The data has daily quantities and prices and discount information (whether the product had a discount and the size of the discount in that given day). This store usually has long discount periods for products (sometimes longer than a month). Almost all products have a discount at some point during the year.

My objective is to at promotions/discounts and see whether it was good or bad for the product. The way I'm approaching this is estimating a flexible machine learning model (I'm using a gradient boosted trees) to predict the quantity sold on a given day:

$$q = F(brand, price, discount) + \varepsilon$$

(I include many other variables such as the weather, flexible dummies for times, trend, characteristics of the product, lagged (one month before) values of price and quantity, etc)

When a product had a discount that day, my counterfactual is predicted with

$$\hat{q}_{counterfactual} = F(brand, price, discount=0)$$

And then the impact of the discount is estimated by the difference between the actual observed value and this counterfactual prediction

$$q - \hat{q}_{counterfactual}$$

1) I make sure the model has a high accuracy in general and then in both the non discount period and the discount period. I'm aware of the general problem of any counterfactual prediction: $$cov(discount, \varepsilon) \neq 0$$, but the hope is that having a flexible enough model will take away from that covariance $$\varepsilon$$. My more pressing concern is that machine learning models might not be consistent for the estimation of partial effects and hence turning off the discount might not give me a good enough proxy of the counterfactual effect. Any ideas on how to approach this?
$$q - \hat{q}_{counterfactual}$$
Would it make more sense to estimate it as the difference between the two predictions, one with $$discount=1$$ and the other with $$discount=0$$?
$$\hat{q} - \hat{q}_{counterfactual}$$