4
$\begingroup$

Let's say I have some data on the amount of money spent on tv ads and total revenue from all sales. The data is available for three separate month.

                       Jan     Feb    March 
$ spent on tv ads      100     110    150
total sales revenue    1000    1000   1500

The problem I am trying to solve is whether the marketing campaign (tv ads) had a significant effect on total sales revenue.

Now, I'm still not sure on how to proceed in terms of tests, either time series analysis, etc. (suggestions?) But one other issue is going to occur, and that regards whether or not to pool the data. Should I run separate tests for each month or do I pool the data together?

So my question is: When is it appropriate to pool data for analysis?

Improve this question
$\endgroup$
0

3 Answers 3

Reset to default
8
$\begingroup$

It's appropriate whenever the elements you're pooling together are homogeneous with respect to the parameters you're estimating. Specifically, this means that, if the model underlying each component is the same, with the same parameter values, then it is fine to pool the data. Otherwise, it's tantamount to leaving an important interaction out of a regression model which can cause misleading inferences.

To place this into the context of your example - if you were to use regression to assess the relationship between $ spent on tv ads and total sales revenue then

If the relationship between $ spent on tv ads and total sales revenue is the same across months, then it is OK to pool them together and ignore the month altogether. But, if the effect does depend on month, then you should potentially include interactions that allow the slope and intercept to vary by month. Failing to stratify your parameter estimates by month when they truly should be can cause misleading estimates of the variable effect(see here for a toy example).

So, the answer to your question boils down to whether you think the effects should vary by month. It seems to me that it probably should, but I'm certainly not as familiar with the application as you are.

Improve this answer
$\endgroup$
5
  • $\begingroup$ How about if the effect varies by month, but is constant for the cross-section? Can we still pool the dataset then? $\endgroup$
    – Mayou
    Commented Sep 6, 2013 at 20:30
  • $\begingroup$ @Mariam, I'm not sure exactly what you mean but it seems, in that case, it would be ok to pool across subsets of the population but not across time. $\endgroup$
    – Macro
    Commented Sep 7, 2013 at 17:15
  • $\begingroup$ following your reasoning, the OP should not have pooled days within months unless the relationship between dollars spent and revenue is the same for every day in the month .... and same for hours within days .... so this reasoning isn’t satisfactory for me. there is obviously no way to verify that the relationship holds for every individual dollar. $\endgroup$
    – user28511
    Commented Mar 14, 2018 at 19:14
  • $\begingroup$ also, your toy example doesn’t imply to me that pooling is misleading. even in the toy example, overall there is no effect. pooling would accurately reflect this. the mistake would be one of interpretation: if you pool, you can’t interpret the data as showing a result that holds for every individual month. all you know is the relationship that holds when all the months are combined (which may be all you need/want). $\endgroup$
    – user28511
    Commented Mar 14, 2018 at 19:18
  • $\begingroup$ i think the real answer is that you should not pool if you have statistical power to find relationships without pooling. when datasets are small, however (as is often the case in real life) pooling is necessary — but limits the precision of the conclusions that can be drawn. $\endgroup$
    – user28511
    Commented Mar 14, 2018 at 19:24
2
$\begingroup$

To add to what Macro said, You can test the assumption of the poolability of the monthly data based on time series analysis. But it would require a lot more monthly data. With the time series analysis you could look for a season trend of a yearly cycle. Even without time series analysis assume several years of data is available you could look for differences between the monthly means and if they appear to be significant then you would not pool the data. The assumption for the test would probably be that the monthly data for any month (say January) is independent from one year to the next. Given a long enough series you could check that assumption to by estimating the year to year correlation.

Improve this answer
$\endgroup$
1
$\begingroup$

In terms of aggregation issues which are related to advertising and sales, you could start with Darryl Clarke and then follow the references forward.

Clarke, Darryl G. “Econometric Measurement of the Duration of Advertising Effect on Sales.” Journal of Marketing Research 13, 4 (1976): 345–57.

Clarke makes the point that the duration of the decay effect seems to interact with the frequency of the measurement in a somewhat uncomfortable way. Weekly data tends to produce decays in a short number of weeks. Monthly data in a short number of months. Yearly data in a short number of years. But I am oversummarizing a complicated field.

Or this overall summary by Tellis (who's a good source for what's happened in this field): http://www-bcf.usc.edu/~tellis/AdGeneral.pdf

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.