0
$\begingroup$

I have two groups of customers. One received a promotion another did not. I want to determine the effect of the promotion. I created a simple linear regression with a dummy variable for weekend.

$test = control + weekend$

This returned an R-Squared of .72 when applied to the pre-intervention data.

When I applied the regression to the post data I looked at the residuals and ran a two-sided T-test to see if they were statistically significantly different from 0.

I was able to reject the null. Now I have a 95% CI for the residuals of 1,000,000 and 1,300,00 and a mean of 1,100,000. My question is, can I conclude that these are unbiased estimators of the effect of the intervention? Or is the interpretation more complicated since the data comes from the residuals of a model with an R-Squared less than 1?

$\endgroup$
3
$\begingroup$

Inferring causality has more to do with experimental design than with R-Squared or goodness of fit. Without knowing much about your data or your particular situation, it sounds like you have at least identified a correlation. To determine whether your independent variable (the promotion) caused the difference in your response metric, you must consider whether your test subjects (customers) were randomly assigned to their respective treatments (promotion vs no promotion). If not, some hidden confounding variable may be responsible for the effect you've observed.

Remember, correlation does not always imply causation.

$\endgroup$
  • $\begingroup$ If the expirement is set up correclty, is there some kind of adjustment that I should make to the CI based on the R-Squared? I feel like the CI should be wider since my model only predicts 72% of the variance of my control group.... If that makes sense. $\endgroup$ – Jarom Sep 1 '17 at 19:57
  • $\begingroup$ Can you briefly explain what your response variable is, along with all of your explanatory variables? If you have an R2 value, at least one of your explanatory variables must be continuous. $\endgroup$ – Anson Call Sep 1 '17 at 20:50
  • $\begingroup$ The response variable is the daily sales volume of a product that had a one-month rebate. The first explanatory variable is another very similar product's daily sales volume that did not have the rebate. The weekend indicator is a dummy variable that is 1 if it is a weekend, 0 if not. $\endgroup$ – Jarom Sep 1 '17 at 23:25
  • $\begingroup$ Ok, thanks. To be honest, I think you need to take a look at your original research question and the methods you're adopting to answer it. From the info you've given me, it sounds like you might need some help creating the appropriate model, interpreting the regression coefficients, understanding residuals and Pearson's correlation, and so on - probably best to consult some other sources and get the personal help you need. I'm not an expert on advertising or marketing, but I think you may be starting with the wrong model. Good luck! $\endgroup$ – Anson Call Sep 2 '17 at 17:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.