Say I want to test the effect of two levels of my independent variable (IV) on some dependent variable (DV). For instance, administering a drug in a small vs large dosage, and then measuring reaction times for each. Assume also that I find that the effect of "high dosage" is greater than the effect of "small dosage".
Does the following argument have any statistical basis, or is it fallacious?
"To make such a result (the effect of this particular drug) more believable, studies are advised to use more than just two levels of the independent variable, e.g. a third intermediary level (medium drug dosage). This is because the finding high>low can happen spuriously with 1/2 probability, whereas the finding high>medium>low can only happen spuriously with 1/6 probability (since 3!=6). Thus, an A>B>C effect is more robust than merely A>B, in terms of proving the effect of the IV (with levels A,B,C) on some DV."
This argument seems sensible intuitively, from a frequentist definition of "chance level" and thus of a Type I error; but it also seems to me wrong, since if a high>low result is statistically significant, it is surely no less significant than the result high>medium>low, if the same alpha level has been used.