2
$\begingroup$

I'm using a Poisson GLM to model the effects of advertising (number of ads bought) on the number of sales (numConvs). If possible I'd like to use the model to get an idea of how many sales are due to one type of ad (ad1), how many are due to another type of ad (ad2), and how many were not caused by either type of ad - a 'baseline' number of sales.

Intuitively I thought I could do this by using the model to predict how many sales I would get when the number of ads bought for one or both types of ad was zero. However the predicted values do not add up:

#===============
# Simulate data
#===============

set.seed(1)
intercept <- 1
ad1Coef <- 0.03
ad2Coef <- 0.05

ad1 <- sample(1:50, size=100, replace=TRUE)
ad2 <- sample(1:50, size=100, replace=TRUE)
numConvs <- rpois(n=length(ad1), lambda=exp(intercept + ad1Coef*ad1 + ad2Coef*ad2))

df <- data.frame(numConvs=numConvs, ad1=ad1, ad2=ad2)

#===============
# Model & predict
#===============

# Model
mPois <- glm(numConvs ~ ad1 + ad2, data=df, family='poisson')
summary(mPois)$coef 

# Predict num conversions based on number of ads in final row of data
finalRowDF <- df[nrow(df),] 
finalRowDF # 43 actual conversions

library('dplyr')
predict(mPois, type='response', newdata=finalRowDF) # Predicted conversions when both ads are playing: 49
predict(mPois, type='response', newdata=mutate(finalRowDF, ad1=0, ad2=0)) # Predicted conversions when no ads are playing: 3
predict(mPois, type='response', newdata=mutate(finalRowDF, ad1=0)) # Predicted conversions when only ad2 is playing: 18
predict(mPois, type='response', newdata=mutate(finalRowDF, ad2=0)) # Predicted conversions when only ad1 is playing: 7

As shown in the code above, the model predicts that there are 49 sales when both ads are playing, 18 sales when only ad2 is playing, 7 sales when only ad1 is playing, and 3 sales when no ads are playing.

From this, I would intuitively think that:

  • there was a 'baseline' number of sales of 3
  • ad2 caused 18 - 3 = 16 sales
  • ad1 caused 7 - 3 = 4 sales

However, these numbers ad up to 23, less than half of the 49 predicted when both ads are playing. What is going on here? I get that a Poisson GLM is a multiplicative model on the response scale, so it's probably not appropriate to use my 'adding and subtracting' method above to partition effects. But even so, where do the remaining 26 predicted conversions come from? Is this some kind of interaction effect, where the effect of ad1 depends on the level of ad2? And is there some appropriate way to get a broad idea of how many ads are caused by ad1 vs ad2 vs baseline effect?

$\endgroup$

1 Answer 1

1
$\begingroup$

In a Poisson model, the expected number of sales given advertising levels $x$ and $w$ would be $$E[y \vert x,w] = \exp(\alpha + \beta \cdot x +\gamma \cdot w ) = \exp(\alpha) \cdot \exp(\beta \cdot x) \cdot \exp(\gamma \cdot w )$$

You can think of $\exp(\alpha)$ as a baseline level of sales without advertising, which is then scaled by the terms involving the ads. If there's no advertising the effect is $\exp(0)=1$ (no effect). In general, however, the effect is multiplicative and definitely depends on the level of the other type.

I think the easiest thing is to examine the ratio of these multipliers $$\frac{\exp(\beta \cdot x)}{\exp(\gamma \cdot w)}=\exp(\beta \cdot x-\gamma \cdot w).$$

One caveat is that the validity of this comparison hinges on the the levels of advertising being uncorrelated with demand shocks, which is often not the case, particularly with internet advertising.

$\endgroup$
5
  • $\begingroup$ Thanks @Dimitriy. I thought the answer might be something along these lines, but that nonetheless it seems strange to me... As I understand it, this means that a Poisson model automatically includes interactions between the terms (insofar as effects of ad1 depends on the effects of ad2). What if you believe that their effects are additive - is this impossible to model with a Poisson GLM? $\endgroup$
    – jay
    Commented Jun 15, 2016 at 1:50
  • $\begingroup$ I have seen a lot of evidence that marketing effects are interactive, so this seems like a feature rather than a bug to me (without knowing more about your setting). If you want to do something additive, why not try OLS? $\endgroup$
    – dimitriy
    Commented Jun 15, 2016 at 1:55
  • $\begingroup$ Agree with you yes marketing effects are interactive, but advertising is just used as an example in this case. The question is really about whether it is possible to partition effects using a Poisson GLM model. It's surprising that a Poisson GLM implicitly models interactions - this isn't something I've ever read/been taught about them. $\endgroup$
    – jay
    Commented Jun 15, 2016 at 1:55
  • $\begingroup$ And just to confirm from your answer, it's impossible to attribute x sales to effects of ad1, y sales to effects of ad2, and z sales to baseline, using a Poisson GLM? $\endgroup$
    – jay
    Commented Jun 15, 2016 at 1:59
  • $\begingroup$ @jay It's not easy to do, but if you want to go this route, I would show three numbers: the $\exp(\alpha)$ no ads baseline, and the predicted value setting each type of advertising to the historic average or median, leaving the other type as is was. $\endgroup$
    – dimitriy
    Commented Jun 15, 2016 at 2:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.