1
$\begingroup$

I have a question about how I can analyze the data from a time series experiment. I'm still a student and therefore am not really familiar with all the procedures and analyzes in SPSS, so i hope someone with the proper knowledge can help me with this problem.

The goal is to show a relation between advertising en sales. To show this we have conducted a experiment with an experimental and control group. The experimental group is a province in which the people gets to see more advertising. In the control group the advertising stays the same as usual.
The dataset contains weekly sales data from several stores throughout the country from the control area and experimental area. And how much advertising is used in all the periods. The data goes back for 1.5 years so there are 70 measures before the experiment, and 10 measures in/after the experiment.

So my question is, how can I prove a relation between those two factors. I know I can do a simple t-test to test the difference between the two groups. But this doesn’t take care of any trends of long/short term effects.

$\endgroup$
5
  • $\begingroup$ You could try changepoint analysis: variation.com/cpa/tech/changepoint.html $\endgroup$ Commented Dec 10, 2013 at 14:42
  • 1
    $\begingroup$ Thanks for your fast response. I read about the Change-Point Analyzer, but because we made use of a control group isn't there a better way than using control charts? $\endgroup$ Commented Dec 10, 2013 at 14:56
  • $\begingroup$ Do you think you have enough data for a mixed-effects model? How many clusters/stores do you have in each region? $\endgroup$
    – charles
    Commented Dec 10, 2013 at 16:06
  • $\begingroup$ @FrankJansen are you interested in the effect of the intervention/experiment or the relationship between advertising and sales? Most of the answers seem to focus on the former rather than the latter. $\endgroup$
    – charles
    Commented Dec 10, 2013 at 18:54
  • $\begingroup$ @charles I hope to show in the first place a significant effect of advertising onsales. If there is a effect i would like to know how big this effect is. i dont think i have enough data for a mixed model, because there are only 6 shops inside the treatment area. in the control area there are plenty. $\endgroup$ Commented Dec 10, 2013 at 19:28

2 Answers 2

2
$\begingroup$

In order to test for the effect of a change in a known variable (advertising spend) one needs to control for both known and unknown impacts that have affected sales. Weekly indicators are certainly known as is the history of the advertising variable. If the series has had a level shift or a change in trend in the history of the data prior to the advertising change then this needs to be empirically identified along with any one time unusual activity and incorporated into the model providing robust estimates of the model parameters. Furthermore there may be autoregressive structure in the tentative model residuals that needs to be addressed in order to render an error process to be Gaussian and any subsequent F or T test to be meaningful. With this in place one can then form a test of the importance of the change in the advertising variable as a Level Shift at the point of the change in advertising would be detected. Unfortunately simple solution tools like SPSS sometimes fall far short of providing this kind of analysis. The literature of the Interrupted Time Series may help you. Search for McCleary and Hay as possible sources to get you started. Unfortunately their work was done in the 80's and is not fully up to date with the modern concepts of Intervention Detection.

EDIT:

Figure 1 from Charles's reference showing a clear level shift(intercept change) and a possible (but probably dubious/insignificant) trend change. Again Charles a thank you for showing the clear need for Intervention Detection procedures.

enter image description here

$\endgroup$
9
  • $\begingroup$ Interrupted Time Series may be a useful approach for this issue. If you want more information this is an easy to read article: archinte.jamanetwork.com/article.aspx?articleid=215841 $\endgroup$
    – charles
    Commented Dec 10, 2013 at 16:09
  • 1
    $\begingroup$ The Figure 1 presentation in your referenced article is a classic case of both a level shift and a time trend change occurring at the same point in time. These conclusions are easily found with software like AUTOBOX (among others) available from autobox.com. I have helped developed that program which uses a General Linear Model approach leading to Robust ARMAX models ensuring that the final error process has constant variance while ensuring invariance of parameters. $\endgroup$
    – IrishStat
    Commented Dec 10, 2013 at 18:27
  • $\begingroup$ @IrishStat Thanks for you response. I was searching for a sollution in the Interrupted Time Series Literature, but my mentor from school said this would be to difficult for my studie. And the time serie (1.5 year) was to short to detect any seasonal effects. Can a program from autobox.com solve this, or can i rather look at other sollutions? $\endgroup$ Commented Dec 10, 2013 at 19:36
  • 1
    $\begingroup$ AUTOBOX can estimate weekly effects (51 in number) with 76 weeks of data BUT the coefficients of the 26 weeks that only had 1 reading might not be as reliable as those weeks with two replications. We normally suggest 3-4 years of weekly data for a weekly model BUT if you only have 1 1/2 years so be it. You can either post 1 of your examples where the ad spend changed or you can contact me off line and I will try and help you and your mentor. $\endgroup$
    – IrishStat
    Commented Dec 10, 2013 at 20:06
  • $\begingroup$ @IrishStat i looked into Autobox. Seems like a really nice piece of software. But I have some questions how it can be helpful to me. 1) I have 6 different stores inside the treatment area, do I have to take like the average of the weekly turnover of these 6 stores or can I include them all? $\endgroup$ Commented Dec 11, 2013 at 8:23
1
$\begingroup$

I think that you could consider using a changepoint analysis. Some other methods that you might consider include:

  1. Split-Plot ANOVA
  2. MANOVA
  3. Mixed Models

In the split-plot ANOVA, we model the data as follows: $$Y_{ijk} = \mu + \alpha_i + \beta_{j(i)}+ \tau_{k}+(\alpha \tau)_{ik}+ \epsilon_{ijk}$$

where $Y_{ijk}$ denotes the sales for treatment $i$ from store $j$ at time $k$. Also, $\alpha_i$ is a treatment effect and $\beta_{j(i)}$ is a random effect of store $j$ receiving treatment $i$. Likewise, $\tau_k$ is a time effect, $(\alpha \tau)_{ik}$ is a treatment-time interaction term, and $\epsilon_{ijk}$ is an error term.

In the MANOVA approach, we would collect the sales data for each store over time and put it in a vector. So let $$\textbf{Y}_{ij} = \begin{bmatrix} Y_{ij1} \\ \ \vdots \\ Y_{ijt} \end{bmatrix}$$

denote the sales data for store $j$ receiving treatment $i$. Doing a MANOVA would test if there is an overall difference between the mean vectors of the two treatments.

The mixed models approach is very similar to the split-plot ANOVA approach except that assumptions are made about the error structure within a single unit.

$\endgroup$
1
  • $\begingroup$ Thanks you very much for your explanation. For this sollution i only need the data after the intevention? Because before the means would probably be the same between the experimental en control group. Furthermore how can i get the is βj(i), (ατ)ik, and ϵijk. Im sorry this is probably a realy dum question, but its been a long time since i worked with SPSS and had the research classes. $\endgroup$ Commented Dec 10, 2013 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.