1
$\begingroup$

My goal is to ascertain whether certain event days are different from the rest of the days. For example, say I want to find out whether pizza-hut sales on days with a pizza-hut advertisement are significantly different from sales on other days. One way I can think of doing this is to have a dummy for the advertisement days and see if it is significant.

However, I need to use the following approach.

(Step 1) Regress pizza-hut sales on total pizza sales (pizza sales of all companies).

(Step 2) Use this estimated equation to compute predicted pizza-hut sales on different days.

(Step 3) Define

excess pizza-hut sales = (actual pizza-hut sales - predicted pizza-hut sales)

(Step 4) On some days excess pizza-hut sales will be statistically significant and on some days they will be not. This would be true whether we focus only on advertisement days (my Group 1) or on non-advertisement days (my Group 2). I want to check whether the proportion of (statistically) significant days in Group 1 is different from proportion of statistically significant days in Group 2.

For this analysis, which I need to carry out I have the following two questions:

(1) When I regress pizza-hut sales on all pizza sales, should I also include a dummy for pizza-hut advertisement days.

(2) Can I run the regression in the same time period for which I need to carry out the above analysis; or do I need to use data from a different time period to estimate the parameters and then use the estimated equation to predict during the time period of interest.

$\endgroup$
1
  • $\begingroup$ Note, what you call "Excess" is what is typically called a residual or error term. Not to be nitpicky, but it is important to be consistent with terminology to avoid confusion. $\endgroup$ Feb 1, 2015 at 4:14

1 Answer 1

1
$\begingroup$

I think you have made this more complicated than it needs to be, while at the same time potentially ignored some really important details to make your modeling results valid. First, the simple model. From you description, you are trying to figure out whether Pizza Hut sales will differ significantly when they advertise. You can use a very straightforward regression model:

$Pizza Hut=\alpha +\beta_1total sales +\beta_2 advertisement$

The model coefficients will tell you:

$\alpha$: the intercept to the model - if you center TotalSales then this will be the average PizzaHut when TotalSales are at their mean, on days without an advertisement. $\beta_1$: the correlation between TotalSales and PizzaHut, assuming no advertisement. If this is statistically significant, and substantively big enough, it will give you the expected relationship (on average) between total sales and Pizza Hut sales. It sounds like your hypothesis is that they are linked. If Pizza Hut is a substantial portion of all sales, you may want to remove Pizza Hut sales from the Total Sales to avoid endogeneity. $\beta_2$: This gives you the average increase in sales as a result of advertising. This is the coefficient you are interested in. If this is significant, it means advertising has a statistical relationship with increased (or potentially decreased, though unlikely) sales. I say a relationship, not an effect, because if advertising corresponds to other things that do not equally effect other pizza shops, those other things may be picked up by the advertising coefficient.

Now, here comes the tricky part. Are your data cross-sectional (do you have a lot of Pizza Huts in different locations in your sample)? Are they time-series (do you have just one Pizza Hut location or a sum of all locations, but data points over time)? Are they panel (you have multiple locations with multiple points of time)? Your answer to this effects how you model the results and what you can say with them. This is important for 2 reasons.

1) Advertising may have a lagged effect: Advertising today may not affect sales until tomorrow. If you have panel or time series data, you want to model this with lags and some form of an ARIMA model, and determine which lag(s) are significant. If you have cross-sectional data, this could mean you may not pick up the effect.

2) Advertising may have a persistent effect: Advertising today may affect sales today, tomorrow, the next day, etc. Same comment as in point 1.

3) Advertising may have an accumulating effect: Advertising today and tomorrow, or advertising every Friday, may have a compound affect. I think this is a common thing in advertising, where the frequency and regularity of advertising is more powerful. There are different ways you could address this, by adding a new variable that represents the frequency or regularity of advertising, an interaction term among advertising days, etc.

4) Sales are likely to be serially correlated. Sales today are going to be related to sales yesterday, a week ago today, a year ago today, etc. This means your observations are not independent, which is a violation of the classical regression hypothesis tests. Same comment as point 1).

5) Sales are likely to be spatially correlated. This means sales in one area are going to be related to sales in a nearby area. If Pizza Hut is well-known and respected for its great pizza, great atmosphere, and great deals in one neighborhood, that may spill over to another neighborhood. Again, this violates the independent observation assumptions. You could cluster observations within geographies, or more comprehensively, use some sort of spatial regression model that includes spatial weights. This is complicated stuff if you haven't been exposed to it, however, so I'd approach this carefully.

I could provide you with detailed modeling suggestions for each of these scenarios, but I believe a deep enough explanation would be more than a response to a single CV post. Others may disagree and provide you greater detail, or if you give some more detail about your data and assumptions maybe we can give you a bit more advise.

$\endgroup$
2
  • $\begingroup$ Thanks for your response. I appreciate it. I am aware of many of the issues you have raised. But I have been tasked to do the test of proportions and need to follow the approach that I have stated in my question. I would appreciate any suggestions regarding the two specific questions I asked. (1) Does it make sense to dummy out the advertisement days -- would that bias the betas invalidating my test of proportions. (2) Should I run the regression in the period of interest or is it advisable to run it in a period before or after the period of interest and why? Thanks. $\endgroup$ Feb 1, 2015 at 14:51
  • $\begingroup$ The problem is it sounds like you are looking for statistically significant observations - that is not possible. You can define substantively-large residuals, which is possible using different leverage statistics, but by definition an observation is not statistically significant; An estimate of a population parameter is. $\endgroup$ Feb 1, 2015 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.