How to test counter factuals

Question

Let's say I have the following data on leads, monthly media spend, and clicks

Month  Leads   Media     Clicks
Jan     150    1000       500
Feb     200    1000       550 
March   300    1200       800
...

Let's say I run a linear regression where y is leads and the predictors are media and clicks. That's good, I know the relationships between these variable and can generate some lags to produce predictions. But what if I had spent 500 (or 2000 or 0) on media, what would have occurred. How do I perform this type of 'counter-factual' analysis where I attempt to find the results of a model if the actual value from one or two of the predictors was lower or higher? What is the standard approach (aka statistically proper approach)? Is it just a matter to "adjusting" the data to the 'new' number and rerunning the regression? or maybe simulating a regression 100+ times with 100+ different values for media?

Answer 1

The simple answer is that you can generate a set of predictor values that looks like the situation you want to know about and then predict the outcome. That's your model's best estimate of what would have happened under that situation.

The complicated answer is that that's one of a few separate approaches of varying complexity and robustness (g-computation, IPW, double-robust estimators like A-IPW and TMLE). There are also considerations of whether or not you have enough covariates to make a meaningful prediction of what would have happened under a counterfactual intervention.

There's a book's worth of knowledge underlying any complete answer to that question. I suggest starting with Hernan and Robins's Causal Inference book here: http://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/