0
$\begingroup$

We are looking at 60 days of a product sales data from a single store, 30 days before and 30 days after the intervention (second display).

We need to evaluate if introduction of second display increased sales.

Is paired sample t-test valid test for this situation?

Data is continuous, there are no outliers in the before / after groups, difference (fist/second/third....etc. day of control period minus fist/second/third....etc. day after intervention) is normally distributed.

Something tells me that paired sample might not be the "correct" test, but a friend from Quality Control tells me that they constantly use paired-samples t-test with the exact same type of data (multiple data points before and after the intervention from same subject)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The assumption of independence is likely to be violated.

This test assumes that each pair of measurements are independent of the other pairs. For example if you had pairs of measurements from several independent shops, i.e. total number of sales the month before and the month after intervention for a single shop.

In your case is it likely that the number of sales today is similar to the number of sales tomorrow? If so, the number of sales 30 days before/after the intervention is not independent of the number of sales 29 days before/after the intervention. This is temporal non-independence and should be accounted for.

To diagnose how important this non-independence is you could make an autocorrelation plot to show how similar close together days are. This question shows an autocorrelation plot with high non-independence.

If it is a problem some kind of additive/smoother/spline regression with an AR1 covariance matrix may be a solution.

Improve this answer
$\endgroup$
6
  • $\begingroup$ If we look on this like matched pairs eg. 1st day pre-intervention vs 1st day post-intervention, 2nd day pre-intervention vs 2nd day post-intervention, which is the way its calculated, it makes somewhat sense. We have 30 subjects (days), measured 2 (pre/post intervention) but….still have the filing that something is not right. $\endgroup$
    – adriano_manu
    Jul 6, 2021 at 13:07
  • $\begingroup$ Thanks for clarifying how it's paired. But the pairs of measurements are still not independent from other pairs, the first day pair is likely more similar to the second day pair than it is to the 30th day pair. $\endgroup$
    – D A Wells
    Jul 6, 2021 at 14:34
  • $\begingroup$ Thanks for the reply! What test would you think that is suitable for this type of data? $\endgroup$
    – adriano_manu
    Jul 6, 2021 at 17:19
  • $\begingroup$ Possibly generalised least squares regression of sales predicted by date and intervention with AR-1 covariance to account for close together dates being more similar. But that could be over kill for your situation, looking at autocorrelation would show how important this is. I would probably not treat the measurements as paired either, unless you believe that the pairs are inherently meaningful in the same way they would be if each pair came from a different store. $\endgroup$
    – D A Wells
    Jul 6, 2021 at 21:18
  • 1
    $\begingroup$ 3. From purely theoretical perspective independence in marketing analysis is practically impossible to achieve. Mixed models are generally preferred. $\endgroup$
    – adriano_manu
    Jul 9, 2021 at 6:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.