Measuring the effectiveness of promotional campaigns (Time Series) I've recently began work as a commercial analyst for a large company after graduating from university 3 months ago with a first class Bsc in Business Economics. On my course we covered a lot of time-series econometric modelling, however I'm not even close to being an expert by any means. I digress:
My manager for an extremely long period of time now has been working on a way to measure the incremental impact of a promotional campaign (i.e. this item was only £1.99 for this period of time, how many sales were driven as a result of this). Unfortunately, there's about a million and one roadblocks that he keeps running in to, as well as the fact he has virtually no history in statistical modelling (which is why he wants my help).
The problems include, let's say item x is on promotion, this almost always means that item y is not on promotion, and vice versa. This needs to be taken in to account, in order to create some kind of benchmark to be used for each product when there is "no promotion" (for example it may look like everyone stops buying item x when it is off promotion, but they have simply moved to item y which is now on promo). The final goal is to have something that the business can use as a tool to try and gauge the effectiveness of a particular promotion should they choose to implement it (so a predictive/forecasting tool).
There are also elements of seasonality (around xmas etc), as well as increases in sales in particular weeks over the year where certain catalogs go out to clients etc.
Essentially just wondering if anybody has any ideas on how we can go about this. Just general brainstorming ideas. How a regression can be set up to try and take this in to account, and what other assumptions will need to be made in order to create a tool such as this.
Thanks
 A: A regression is already a very good idea. You will need to collect as much data as you can, then encode the relevant pieces of information into regressors, estimate the model, and finally interpret the model. You already mention quite a number of drivers that you will need to include:


*

*Promotions & price changes

*Calendar events, like Christmas (I like to model ramp-ups to holidays)

*Catalog drops

*Cannibalization


A couple of other things you should really think about:


*

*Are your sales data constrained? If a product is not available during part of the year, zero sales don't necessarily imply zero demand. Some people like to infer demands during such periods - I personally believe that inference in such a situation simulates more knowledge than we have, and like to remove such data points.

*Same thing for supply-restricted sales ("censored" data). If a warehouse had 100 units and sold 100 units, you may have had to turn away customers. Think about how to model this.

*Seasonality, and lifecycle effects. I personally like to model these using Gaussian or other dummies.

*Depending on what kind of product you sell, consider "pantry-loading" behavior, when end customers or B2B customers load up on product during promotions and you see a post-promotion dip. This notoriously happens if drugstores reliably put washing powder on promotion every three weeks - essentially, people will stop buying during non-promotional weeks, so your estimate of the baseline is pretty much worthless.


You will need to think about how to validate such a tool. I personally like to use the tool for forecasting and evaluate it on a holdout sample, reasoning that if it doesn't improve on simple benchmarks, it likely doesn't understand the dynamics. However, beware of selection effects: if you analyze a bunch of potential promotions and finally only run those that forecast the highest return, you will almost certainly get performance below what you predicted, because of regression toward the mean.
In terms of models, since you may have cross-product effects, you may want to look at vector autoregressions with external effects, or VARX models. Or you may want to look at third-party tools, since this is a quite non-trivial question with potentially large monetary payoffs, which people do throw significant manpower at. (E.g., this tool, which I work for.)
A: You can consider an appropriate regression model which will be able to handle various features of your data.
I would suggest generalized least squares, a type of regression which allows for correlation between the residuals. Given you are working with time series data it is very likely your residuals will be correlated across time (i.e. periods of unexpectedly high sales are likely to be followed by more periods of high sales).
Within this regression model I would suggest the following which address your various requirements:


*

*Cannibalization by Y: create dummy variables for when item Y is being discounted.

*Discounts for X: create dummy variables for when item X is being discounted.

*Seasonality: an easy way to deal with this is to create dummy variables for each month, and maybe also dummy variables related to if it is just before a major holiday.


From this model you will be able to estimate the effect which putting item X on sale has on its sales (based on the coefficient attached to the X discount dummy variable). Also by fitting an appropriate time series model to the residuals (possibly an ARMA model) you could forecast future sales.
