2
$\begingroup$

I have 43 weeks of data (here) that show the resulting weekly sales of a group of test stores which received a treatment and sales of other "control" stores that did not. There were 36 test stores and 525 control stores - assumed to be very similar to the test stores.

The treatment began in week 41. What is the resulting lift in dollars and % - and in what range?

I assume I should use regression to solve this, but I am not really sure of the correct way to do that. I have been looking at using CausalImpact in R, but I am not sure I trust the results. Thanks in advance for your help.

Improve this question
$\endgroup$
4
  • $\begingroup$ If indeed the control stores were "very similar" to the test stores, then why are their total weekly sales almost the same despite having ten times as many control stores? Something's not consistent in your description. For the best analysis of this situation, you ought to examine the actual weekly sales of all 110 stores, not just their totals. $\endgroup$
    – whuber ♦
    Commented Dec 22, 2017 at 15:56
  • $\begingroup$ My bad - I pasted the average sales. I edited the question. Will I get better results if I use actuals instead of averages? $\endgroup$
    – Russ Harris
    Commented Dec 22, 2017 at 17:16
  • $\begingroup$ The advantages of tracking actual sales by store include (but are not limited to) (1) being able to assess the distribution of sales across all stores at each week and (2) being able to assess the correlations between sales, by store, from week to week. This makes powerful techniques like time series analysis available to you. $\endgroup$
    – whuber ♦
    Commented Dec 22, 2017 at 17:19
  • 1
    $\begingroup$ OK - I went ahead and dumped detail for all stores - not just the 10 test stores I was originally concerned with. Detail does not contain averages. THANKS FOR YOUR HELP! $\endgroup$
    – Russ Harris
    Commented Dec 22, 2017 at 17:43

1 Answer 1

Reset to default
1
$\begingroup$

You say ..."I assume I should use regression to solve this" . I would say "NO" due to the fact that the data is auto-correlated . http://autobox.com/dave/regvsbox.pdf (which I authored) discusses issues/differences/opportunities/pitfalls when dealing with time series that your possible regression solutions may be ignoring.

Edited after receipt of the detailed data 561 stores .... 52 observations per store.

The more I see of the details of 561 (525+36) stores over 52 weeks, the less I am comfortable about analyzing the details (52x561 values). The actual sales for the test group for periods 41-52 could be compared to the expected average sales for that period based upon the actual historical sales of the control group (1-40) and the actual sales for the control group for the period (41-52) using a regression-type approach.

Please post the average sales for the two groups for periods 1-52 as a two column csv file.i.e. 104 values and I will try to help.

As much as I normally ask for all the details , I am at a loss (at the present time) as to how to proceed with a detailed analysis of the 561 samples each with 52 measurements while incorporating temporal effects et al. It smacks of a multi-equational vector ARIMA (561 endogenous series) problem with an exogenous indicator reflecting control vs test and of course potential provisions for unspecified but possibly present and detectable deterministic effects such as pulses/level shifts/local time trends.

Improve this answer
$\endgroup$
1
  • $\begingroup$ If there is autocorrelation anywhere/anytime that needs to be conditioned for in any analysis. I am confused by your reflection '"should be expected " . Auto-correlation deals with internal (auto) correlation within a series not cross-correlation which is between two series. $\endgroup$
    – IrishStat
    Commented Dec 22, 2017 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.