chi square tests & graphing

Question

I was wondering what the appropriate test would be for comparing the results of two chi-square tests.

For example: I have one experiment where I determined if there was a significant difference in the ability of rats to detect the smell of an odor (odor 1) compared with a control (yes/no) using chi-square. I then repeated the experiment but using a different odor (odor 2 vs. control).

Now I want to compare the results of those two tests statistically (results of experiment 1, odor 1 vs. results of experiment 2, odor 2), what test would I use?

I heard Mann-Whitney $U$ might be good, but I need to justify why this test would be better than other test for this comparison.

Also, how would I graph the data with error bars, or are error bars not used for chi-square?

Answer 1

It's a little difficult to work out exactly what is going on here. Perhaps this: you have a 2 x 2 classification: rats can/cannot identify a control/odor 1, and you are counting the 4 frequencies; and again for control/odor 2.

Given two chi-square tests, you can compare them in terms of the chi-square statistics and the resulting $P$-values and perhaps look at residuals. If the number of rats differed between the experiments, that would be problematic.

I can't see any sense in which there is a test to compare the two tests. What probability model do you have in mind for that?

Simply, the tests are just different. To compare them most easily, the rats must be the same and the conditions must be independent. For example, the rats must be completely recovered from any side-effects; also, it is to be assumed that there are no learning or memory effects.

If you have a record of which rats reacted in which way, you could recast the analysis in terms of a 2 x 3 table, can/cannot identify control/odor 1/odor 2. But at a minimum it must be the same rats, all classified on all the possibilities.

I can't see any sense in which Mann-Whitney $U$ applies to anything you mention. Is that idea just gossip or are you reacting to a textbook or explanation of methods?

Naturally you may have further data but there is no point in speculating on that.

Some people do draw graphs of the 4 frequencies for a 2 x 2 table, but it's hard (for me) to see what that gains you. (Elsewhere I am enormously fond of graphics.)