Extracting the effect of TV ad campaign on time series with seasonality

Question

I was wondering what would be the best way to measure the effect of a TV ad campaign on sales (phone and online) and other metrics such as website visits.

One could argue that most of the effect of the ad would be in the next few minutes/hours, but it could linger for a couple days too.

Setup

• Let's say I have the counts of visits/ sales by period of five minutes for two years.
• Let's say that there were two ad campaign every year and that they were lasting one month each, with multiple ads running everyday.
• Let's say we have the date, time and channel that each ad was run.

Questions

1. How would you measure the impact of the ads on the sales/visits assuming that there is seasonality over the year and during the day?
2. How would we go to find on which channel/ time of day/ time of year the ads have the most impact?

Answer 1

There are several approaches to tackle this issue. A straightforward and easy way is by splitting the year in buckets (e.g. 4 buckets, one per season) and do the same with day (e.g. 4 buckets, morning, afternoon, evening, night). These are indicative buckets, you can make more/less buckets if you want.

Then you can create dummy variables per each bucket and apply standard OLS regression of Sales against Cost of ad for example (you can choose whatever you want here) and these dummy variables. So what you do is to control for each season/period of time.

$Y_{t} = \beta_{0} + \beta{1}D_{2}+ \beta{2}D_{3}+ \beta{3}D_{4} + \epsilon$

Where $D_{k}$ corresponds to each season. As you may notice you can see that we omit $D_{1}$ as we take it as base category, so we can compare the other coefficients in relation to that one. If you included all $D_{k}$ you'd have a problem of multicollinearity.

You can find here an example of what I explained. We regress Ice cream sales against Temperature but then we control for seasonality and the effect it's clearly smaller.

$Y_{t} = \beta_{0} + \gamma Temp + \beta{1}D_{2}+ \beta{2}D_{3}+ \beta{3}D_{4} + \epsilon$

To know when there is a greater impact look at $\beta$ coefficients.

I hope this (basic but useful) approach helps you!

Answer 2

I'm not convinced that seasonality should be your primary concern. Attribution of advtg impact is a controversial minefield and one about which the industry has not come to anything even close to agreement. See this article for confirmation of that... http://mashable.com/2012/07/26/ad-attribution-model/#CpfzyOS4Ekq3

I would start with a basic panel data model and gradually introduce greater complexity. The basic model could be built using several possible functional forms as described in this book on this class of marketing science models... http://www.anderson.ucla.edu/faculty/lee.cooper/MCI_Book/BOOKI2010.pdf The options include linear, multiplicative and exponential with very different shapes and consequences in terms of how your marketing instrument (advg) is related to subscription (the target variable). This book is a veritable cookbook of suggestions about how to do that.

Next, there are so many ways of exploring, decomposing and/or cumulating the advtg measures, for instance, the literature talks about adstock metrics. These are lagged assumptions about how marketing activities decay over time. There's a big literature about this, e.g., here ... https://mpra.ub.uni-muenchen.de/7683/4/Adstock_Model.pdf

As I see it, the biggest issue you face is integrating website visits with digital ad exposure, both of which lead to subscription. Hopefully you have the advtg in some standard or consistent metric (across channels) like GRPs or TRPs by channel and time. Then, you can explore how these relate in some aggregate sense in driving website activity and subsequent subscription. It's not clear to me how you can get this to work and, as noted above, your attribution model choice may be the determinant of that.

The good news is that there is a big marketing science literature out there that deals with these issues. The bad news is that no one agrees on the best way to do anything. This means that, whatever you end up doing, you have to recognize that it will be subject to challenge, particularly from sceptical clients. So, just be prepared to motivate your choices and models.