1
$\begingroup$

I'm trying to measure the impact of a promotional activity and want to see whether it is contributing to business performance.

Let's say there are 100 campaigns. For some campaigns, there is no promotional activity at all. If a campaign has promotional activity, the % of items included in the campaign might differ.

I used '% of items included in the promotional activity' as my independent variable and revenue as my dependent variable. The results show that promotional activity is insignificant with a p value of 0.7 and the coefficient is negative. Also tried binary categorization (promotion vs no promotion) and the results are similar. My R squared values are close to zero.

I'm aware that this is a very limited analysis so far. But I'm quite new to this and any comments would be highly appreciated. To understand the impact of an activity, do I need to build a whole model that explains the variance correctly?

  • What else should I do to improve it?
  • Should I add more variables, such as number of # of visitors, into regression and why?
  • Would running the same regression on subsets, like only campaigns in category x, make sense to see if promotion had an impact on that specific campaign? Would I gain anything by disaggregating data on the subsets above?
$\endgroup$

1 Answer 1

1
$\begingroup$

What else should I do to improve it?

Maybe the association is nonlinear. Did you plot the data first ?

Should I add more variables, such as number of # of visitors, into regression and why?

If you have other variables that are potentially associated with the outcome then yes. Why ? Because it could improve the model. But first try to look at all the variables and consider how they may be related to each other. Draw a DAG and use this to identify potential confounders or competing exposures, which should be included, and mediators (which should not). What you want to do is include as many, but not too many, variables, as possible. You seem to be interested in causal effect (inference). The DAG will tell you which variables to include and which not. A good tool for this is dagitty (www.dagitty.net). If you don't have knowledge of the possible causal structure among your variables, then you have what's called a variable selection problem. For prediction this is not too problematic, but for inference it can be a big problem, because you run the risk of over-adjustment or including mediators which can severely bias the results.

On the other hand, if you include only 1 variable to explain your outcome variable, then the association could be confounded.

Would running the same regression on subsets, like only campaigns in category x, make sense to see if promotion had an impact on that specific campaign?

Seperate models for groups is usually a bad idea, as you then lose statistical power to detect associations

Since you have groups (categories) it is possible that there are correlations within each group. This can be addressed by fitting random intercepts for groups, in a mixed effects model. . Alternatively if you have a small number of categories then you could include the category variable as a fixed effect.

Would I gain anything by disaggregating data on the subsets above?

I assume you mean aggregating, not disaggregating. No, this leads to loss of information and reduced statistical power.

$\endgroup$
4
  • $\begingroup$ Thank you Robert. I will ask a silly question. If you want to understand the impact of an action on one of your metrics, do you need to create a whole model for the metric? I'm asking this because, if you need to model 'what affects sales?', that's a huge task and there are many factors that I can think of that would be useful. But I just want to see if this promotion works or not. Is modelling sales the only way? $\endgroup$ Jan 1, 2020 at 17:19
  • $\begingroup$ I have updated the question to address this. I hope it makes sense. $\endgroup$ Jan 1, 2020 at 17:42
  • $\begingroup$ thank you very much Robert. I have one final question to ask: let's say after doing all the work, I've identified 5 variables (some categorical, some continious). I also add my binary 'promotion' variable, 1 if the campaign had a promotion, 0 otherwise. I run a regression on sales. How would I interpret the results? Do I simply check the categorical variable p-value and coefficient (because I'm already controlling for the other 5)? $\endgroup$ Jan 2, 2020 at 17:22
  • $\begingroup$ Yes that is correct $\endgroup$ Jan 2, 2020 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.