I am studying the effects of a sales program on the weekly unit sales at ~1,000 retail locations. I am having trouble figuring out which statistical test is appropriate to run for this scenario. Here is the background of my problem.

I have ~1,000 stores which are divided into 3 groups:

  • Program Group (~600 locations) - had a sales program applied
  • Control Group (~400 locations) - did not have any program applied
  • Online Group (1 location) - online sales without physical location

Each store has 17 weeks of data. Each week had one of these three possible conditions applied. These conditions did not run consecutive and they not overlapping.

  • Program A: 5 out of 10 weeks
  • Program B: 2 weeks out of 10 weeks
  • Null Program (no program): Remaining 10 weeks

Only stores in the program group were subjected to one of the conditions and all ~600 stores were subjected for the same program for that given week. There is no reason to believe that a program would impact sales at Control stores and should be considered isolated.

The dependent variable being measured here is sales units for that week. The specific question is whether Program A or Program B or both had a measurable impact on sales against the control group. Additionally, it is possible that the program had an indirect effect on online sales which is why it is included separately.

I think an ANOVA test is required for this. However I got confused because I have repeated measures with multiple conditions. Should I be considering all 17 measures for each location (17 x 1,000)? Or do I aggregate numbers? I'm not sure how to get started. If someone can point me in the right direction it would be helpful. I have R and Excel I can use to run this analysis.


Your best approach is to use a linear mixed effects model -- it's basically a repeated-measures ANOVA, but has some advantages, like letting you include time in your model and dealing with your two different groups nicely.

Use the lme4 packaged in R.

Since your program group included the same null condition as your control group, I'd include just a prog designation, rather than separating the groups:

store   prog    week    sales
a       A       1       94
a       A       2       124
a       A       3       297
a       A       4       400
a       A       5       395
a       B       6       528
a       B       7       35
a       N       8       80
a       N       9       90
a       N       10      100
b       B       1       2
b       B       2       112
b       A       3       207
o       N       7       71
o       N       8       94
o       N       9       106
o       N       10      107
p       N       1       29
p       N       2       34
p       N       3       49
p       N       4       47
p       N       5       63
p       N       6       69
p       N       7       72
p       N       8       83
p       N       9       95
p       N       10      117

So, some stores will have prog A, B, and N and some will just have N.

Then you can predict sales depending on the program, with store as the random effect, as follows:

mod = lmer(sales ~ prog + (1|store),data=a)

I'd also include the week (which should be numeric):

mod = lmer(sales ~ prog + week + (1|store),data=a)

Your output will give you the slope estimates for A vs. B (progB is the slope between A and B) and for A vs. N (progN).

Fixed effects:
            Estimate    Std. Error     df t value   
(Intercept)   142.83      25.44  156.00   5.615
progB         -11.59      32.49  156.00  -0.357
progN        -189.53      21.66  156.00  -8.750
week           20.40       3.38  156.00   6.036

P values are tricky to get with lmer, so if these are really important to you, I'd suggest reading up on why they are tricky. If you just want an estimate of what's working and what's not, then I suggest using another package lmerTest to give you these values:

Fixed effects:
            Estimate Std. Error      df t value Pr(>|t|)    
(Intercept)   142.83      25.44  156.00   5.615 8.80e-08 ***
progB         -11.59      32.49  156.00  -0.357    0.722    
progN        -189.53      21.66  156.00  -8.750 3.11e-15 ***
week           20.40       3.38  156.00   6.036 1.11e-08 ***

So in my example, there's no difference between A and B, but there is between A (and thus also B) and N. This effect is above and beyond the effect of week (which is also driving sales). Depending on how this works out, you may want to shift around your condition names or do follow up tests just to make sure that B is more effective than N as well, since that doesn't fall out of here.

I'd do all of this first, then worry about the online store. If the programs were run in the same order in all stores (which I did not assume before), then a simple linear model should work:

mod = lm(onlineSales ~ prog + week)

(If all the stores had a different order, I think it will be hard to establish whether the programs had any effect at all, though I guess you could say "When 80% of the stores were implementing A, online sales went up...")

  • $\begingroup$ Thank you for this. I'm about to give it a shot. One question, the store column you have {a,b,...,o,p}. I only have three groupings of store (program, control, online). Where did you get the 4 letters from? Also, it looks like you are aggregating the individual store sales to a group total? $\endgroup$ – ElPresidente May 6 '15 at 18:34
  • $\begingroup$ Oh, that was just my quick pass of labeling a store (to make sure they get read in as a factor, not numeric). 'a' is store #1, 'b' is store #2. So no, the sales correspond to individual store sales. $\endgroup$ – Amyunimus May 6 '15 at 18:45
  • $\begingroup$ That's what I thought, but wasn't sure. Thanks. I will read up on the package and see if this works with my data. $\endgroup$ – ElPresidente May 6 '15 at 18:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.