1
$\begingroup$

I am trying to refine the way my company validates tests in retail stores for products that we sell. The prior way was only to look at immediate change in dollar and unit sales without taking into account what would have happened without the test.

The direction I am headed is A/B testing with holdouts that have similar directional sales over the last 52 weeks. There would be a post and pre period to compare the test vs control. My biggest roadblock is trying to convince the team and myself that a Pearson correlation time series on sales is not the correct way to identify a set of control stores for the test.

We want to test anywhere from 50 to 2000 test stores at a time and try and obtain a 1:3 test to control store ratio with offset test dates. I really think we should cluster the stores using time series and choosing the stores with the closest means but am having a hard time convincing others. We would be either using R or even Aster R.

I have also used the R package CausalImpact which seems to be a work around for now but for ramping up to a large scale continuous production testing environment am not sure that will suffice.

In summary what is the best way to identify control stores to match test stores on data sets that can be in excess of 3 million records?

$\endgroup$
0
$\begingroup$

You say " My biggest roadblock is trying to convince the team and myself that a Pearson correlation time series ...... " . I say " You might want to see my answer and my comments to this question as I raise serious challenges to the " pearson correlation coefficient is used when time series is being analyzed" Correlation of non-stationary time series

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.