We manufacture foobars. In July, 91% of foobars were defect-free, but in August that figure was 89%.

Would chi-squared be the right method to determine if the difference of 2% between July and August is significant? I read somewhere that chi-squared should not be used to compare same variable from different time periods, as it would not qualify as independent variable in this case.


On prompt from @Glen_b, I dug out the passage where I got this idea from: "What is a p-value anyway?" by Andrew Vickers, page 196.

In answer to a review question, the book says: "All of the common statistical tests assume that the data are independent. Applying these tests to non-independent data is a very common error. An obvious example is repeat observations." The example data follows, in which sales over a week in two different stores are split into daily sales, so that instead of comparing two data points (weekly sales in each store) we can compare 14 data points (daily sales in each store). The erroneous assumption here is that we have 14 independent data points, while in fact we have two sets of non-independent data points, thus t-test or chi-squared cannot be applied.

The author defines independence as: "...two variables are independent if information about one gives you no information about the other." He then says this about the given example: "...if I tell you Monday's sales figures, you can take a reasonable guess at Tuesday's". Therefore, Tuesday sales are not independent from Monday sales.

Similarly, in my example August measurements are not independent of July measurements.

  • 1
    $\begingroup$ Can you say where you read this? Some context would help. Can you explain why you have used the 'paired comparisons' tag? Where does the pairing arise? $\endgroup$ – Glen_b Aug 12 '13 at 2:05
  • $\begingroup$ @Glen_b, I don't remember exactly where, possibly in this book. Not so much pairing, but the independent property of the variable is in question here. It arises from the fact that it is the same measurement, arbitrarily split into two time periods. In other words, the August measurement is somehow linked to July measurement. I don't know if this is sound reasoning, hence the question. $\endgroup$ – Dimitri B Aug 16 '13 at 2:36
  • $\begingroup$ So dependent in some sense, rather than specifically paired, then? Perhaps either the dependence or the non-independent tag is a better fit? (I am not sure why there's two) $\endgroup$ – Glen_b Aug 16 '13 at 3:12

Two sample proportion test is appropriate here.

| cite | improve this answer | |
  • 1
    $\begingroup$ two sample proportion z-test requires that two samples are random and independent of each other. Considering that we split the same foobars into "before" and "after" an arbitrary event (start of new month), are these samples random and independent? $\endgroup$ – Dimitri B Aug 16 '13 at 3:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.