3
$\begingroup$

I was wondering what would be the best way to measure the effect of a TV ad campaign on sales (phone and online) and other metrics such as website visits.

One could argue that most of the effect of the ad would be in the next few minutes/hours, but it could linger for a couple days too.

Setup

  • Let's say I have the counts of visits/ sales by period of five minutes for two years.
  • Let's say that there were two ad campaign every year and that they were lasting one month each, with multiple ads running everyday.
  • Let's say we have the date, time and channel that each ad was run.

Questions

  1. How would you measure the impact of the ads on the sales/visits assuming that there is seasonality over the year and during the day?
  2. How would we go to find on which channel/ time of day/ time of year the ads have the most impact?
$\endgroup$
  • $\begingroup$ Is this only for 1 or more than 1 product? Is sales in units or dollars? How do you propose to deal with attribution issues? How are the ad campaigns measured? GRPs? TRPs? What does "channel" refer to? Is the marketing both online and off? $\endgroup$ – DJohnson Oct 31 '16 at 15:55
  • $\begingroup$ * Let's go for 1 product and sales per unit. Attribution is exactly what I am trying to do here, sorry if that wasnt clear. Ads are measured as in the tv spot was aired at this time ,that time and that other time on the news channel and another time on the weather channel. Channel as in Tv channel. Let's go for ofline only, as online is easier to attribute. $\endgroup$ – Zoltan Oct 31 '16 at 16:30
  • $\begingroup$ Another question: what is the unit of analysis? An individual as expressed by a credit card number? Or the transaction? If the former and assuming that multiple sales are possible over the two year span, then a hierarchical marketing mixture model is possible. If the latter, then a simpler model would be required. $\endgroup$ – DJohnson Oct 31 '16 at 17:10
  • $\begingroup$ hi again and thanks for replying! It is a product that you subscribe to, so only one sales per customer ( unless they churn and return, but maybe we could assume they cant do that for now?). I will look up hierarchical marketing mixture, this is a completely new field for me. $\endgroup$ – Zoltan Oct 31 '16 at 21:22
1
$\begingroup$

There are several approaches to tackle this issue. A straightforward and easy way is by splitting the year in buckets (e.g. 4 buckets, one per season) and do the same with day (e.g. 4 buckets, morning, afternoon, evening, night). These are indicative buckets, you can make more/less buckets if you want.

Then you can create dummy variables per each bucket and apply standard OLS regression of Sales against Cost of ad for example (you can choose whatever you want here) and these dummy variables. So what you do is to control for each season/period of time.

$Y_{t} = \beta_{0} + \beta{1}D_{2}+ \beta{2}D_{3}+ \beta{3}D_{4} + \epsilon$

Where $D_{k}$ corresponds to each season. As you may notice you can see that we omit $D_{1}$ as we take it as base category, so we can compare the other coefficients in relation to that one. If you included all $D_{k}$ you'd have a problem of multicollinearity.

You can find here an example of what I explained. We regress Ice cream sales against Temperature but then we control for seasonality and the effect it's clearly smaller.

$Y_{t} = \beta_{0} + \gamma Temp + \beta{1}D_{2}+ \beta{2}D_{3}+ \beta{3}D_{4} + \epsilon$

To know when there is a greater impact look at $\beta$ coefficients.

I hope this (basic but useful) approach helps you! Example

$\endgroup$
1
$\begingroup$

I'm not convinced that seasonality should be your primary concern. Attribution of advtg impact is a controversial minefield and one about which the industry has not come to anything even close to agreement. See this article for confirmation of that... http://mashable.com/2012/07/26/ad-attribution-model/#CpfzyOS4Ekq3

I would start with a basic panel data model and gradually introduce greater complexity. The basic model could be built using several possible functional forms as described in this book on this class of marketing science models... http://www.anderson.ucla.edu/faculty/lee.cooper/MCI_Book/BOOKI2010.pdf The options include linear, multiplicative and exponential with very different shapes and consequences in terms of how your marketing instrument (advg) is related to subscription (the target variable). This book is a veritable cookbook of suggestions about how to do that.

Next, there are so many ways of exploring, decomposing and/or cumulating the advtg measures, for instance, the literature talks about adstock metrics. These are lagged assumptions about how marketing activities decay over time. There's a big literature about this, e.g., here ... https://mpra.ub.uni-muenchen.de/7683/4/Adstock_Model.pdf

As I see it, the biggest issue you face is integrating website visits with digital ad exposure, both of which lead to subscription. Hopefully you have the advtg in some standard or consistent metric (across channels) like GRPs or TRPs by channel and time. Then, you can explore how these relate in some aggregate sense in driving website activity and subsequent subscription. It's not clear to me how you can get this to work and, as noted above, your attribution model choice may be the determinant of that.

The good news is that there is a big marketing science literature out there that deals with these issues. The bad news is that no one agrees on the best way to do anything. This means that, whatever you end up doing, you have to recognize that it will be subject to challenge, particularly from sceptical clients. So, just be prepared to motivate your choices and models.

$\endgroup$
  • $\begingroup$ Wow, thanks for this detailed reply! I now have a lot of reading to do and new keywords tomsearch for, eill report back when I am done. $\endgroup$ – Zoltan Oct 31 '16 at 22:46
  • $\begingroup$ Hi DJohnson,I gave a quick look at Lee cooper's book, and it is showing tons of models for market share of a product. How do you use that for attribution? thanks $\endgroup$ – Zoltan Nov 22 '16 at 1:35
  • $\begingroup$ @Zoltan Forget that it's focused on market share. It's simply one of the best introductions to marketing mix modeling out there and MMMs are at the heart of attribution. How you choose to integrate the attribution component is reviewed in the first link in my response (above). $\endgroup$ – DJohnson Nov 22 '16 at 2:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.