Measuring impact of 1 independent variable on 1 dependent variable

Question

I'm trying to measure the impact of a promotional activity and want to see whether it is contributing to business performance.

Let's say there are 100 campaigns. For some campaigns, there is no promotional activity at all. If a campaign has promotional activity, the % of items included in the campaign might differ.

I used '% of items included in the promotional activity' as my independent variable and revenue as my dependent variable. The results show that promotional activity is insignificant with a p value of 0.7 and the coefficient is negative. Also tried binary categorization (promotion vs no promotion) and the results are similar. My R squared values are close to zero.

I'm aware that this is a very limited analysis so far. But I'm quite new to this and any comments would be highly appreciated. To understand the impact of an activity, do I need to build a whole model that explains the variance correctly?

• What else should I do to improve it?
• Should I add more variables, such as number of # of visitors, into regression and why?
• Would running the same regression on subsets, like only campaigns in category x, make sense to see if promotion had an impact on that specific campaign? Would I gain anything by disaggregating data on the subsets above?

Answer 1

What else should I do to improve it?

Maybe the association is nonlinear. Did you plot the data first ?

Should I add more variables, such as number of # of visitors, into regression and why?

If you have other variables that are potentially associated with the outcome then yes. Why ? Because it could improve the model. But first try to look at all the variables and consider how they may be related to each other. Draw a DAG and use this to identify potential confounders or competing exposures, which should be included, and mediators (which should not). What you want to do is include as many, but not too many, variables, as possible. You seem to be interested in causal effect (inference). The DAG will tell you which variables to include and which not. A good tool for this is dagitty (www.dagitty.net). If you don't have knowledge of the possible causal structure among your variables, then you have what's called a variable selection problem. For prediction this is not too problematic, but for inference it can be a big problem, because you run the risk of over-adjustment or including mediators which can severely bias the results.

On the other hand, if you include only 1 variable to explain your outcome variable, then the association could be confounded.

Would running the same regression on subsets, like only campaigns in category x, make sense to see if promotion had an impact on that specific campaign?

Seperate models for groups is usually a bad idea, as you then lose statistical power to detect associations

Since you have groups (categories) it is possible that there are correlations within each group. This can be addressed by fitting random intercepts for groups, in a mixed effects model. . Alternatively if you have a small number of categories then you could include the category variable as a fixed effect.

Would I gain anything by disaggregating data on the subsets above?

I assume you mean aggregating, not disaggregating. No, this leads to loss of information and reduced statistical power.