I am working on a problem where I will be using a hypothesis test to find the difference in two means; a control group, and a test group. Each group consists of several stores where observations are taken every week. Imagine the dataset as so: data set

As you can see, I have 5 stores in my control group, and 3 stores in my test group. And my formula is as follows (hypothesis test for independent samples, population variance unknown): t formula to test the differnce in means assuming equal variance

My question is... for my N.. Should I use 5 and 3? Or should I account for the number of observations for each store as well? (I.e. N = 15 & 9). Does it matter?



You use the number of observations taken, since that is how you increase the confidence in your estimate. Go with $15$ and $9$.

However, an assumption of the t-test is that the observations are independent of one another, which you violate. The extent to which you believe this matters will affect how you proceed. There are more advanced modeling techniques that will account for this. If you are part of a business that is investigating this, that will require you to hire (or contract with) a statistician. In the more likely scenario that this is part of an assignment for freshman business analytics, then the right answer is $15$ and $9$.

  • $\begingroup$ Thank you for your answer... My understanding of hypothesis testing for related populations shaped my assumption that my example the two groups are independent. Allow me to further clarify... If I measured one group of stores in a before/after scenario, this would be considered a dependent or related sample. Because I have two separate groups of stores (note the store ID's do not overlap), I assume these are independent. Is my thinking off? $\endgroup$
    – Snowy
    Nov 19 '20 at 22:17
  • $\begingroup$ @Snowy That's called a paired test. You do a one-sample test on the after-before differences. $\endgroup$
    – Dave
    Nov 19 '20 at 22:20
  • $\begingroup$ thanks... I guess I'm still uncertain on how two groups of different stores are not independent of one another. Could you explain more? Or perhaps mention some of the more advanced techniques that would account for this and I can do some research. $\endgroup$
    – Snowy
    Nov 19 '20 at 22:52
  • $\begingroup$ Measurements in store 1005 are related in a way that they are not related to measurements from store 1008. That is how you violate the assumption of independence. $\endgroup$
    – Dave
    Nov 19 '20 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.