1
$\begingroup$

The section "Bonferroni Comparisons" in the book Applied Statistics for Engineers and Scientists, Petruccelli at el. (1999), the authors wrote:

Bonferroni intervals are ideal for making a small number of pre-specified comparisons (i.e., comparisons decided on before looking at the data).

I don't understand how the comparisons can be made without looking into the data (= it means data snooping?)? I mean without data how one can even construct the interval?

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, prespecified comparison can easily be misunderstood. What is meant here should be seen in a situation where you can make many tests with the same data. The most common case is: you have two samples, and many variables measured on them, and you want to test if the samples come from the same population. Then you could make a separate test on each of the variables.

For example, you measure 100 variables like height, weight, bmi, age, ... on people from town A and town B and want to check the null hypothesis, that it does not matter if you come from town A or B.

Then it is not correct to make 100 tests and say, if one of them is significant, then I conclude that A and B are different. For example, if you set significance level to 5%, you would on average in such a situation have 5 tests that are significant, even if A and B are in fact the same population.

The Bonferroni correction takes care for that by adjusting the significance level for each of the $m=100$ tests. It requires that you have to have at least one test with a p-value of $\alpha/m$ in order to be allowed to say that A $\neq$ B. In our example, one of the 100 tests would need to reach $p \leq 0.0005$!

To avoid that you need one test with a very low p-value, it is a good idea to work with fewer tests. Selecting these tests in advance is meant by "prespecified comparisons". One should typically look at the scientific question and decide for which of the 100 variables you would expect a difference at all, if the populations of A and B are not the same.

In practice, people often look at the data in advance, but this is not correct methodology. Then the comparison are no longer prespecified.

$\endgroup$
7
  • $\begingroup$ thanks for your answer! However, the example is beyond my grasp. It would be nice if you explain it using the one-way model which I'm studying, or some resources that I can elaborate on are more than enough. $\endgroup$
    – Tran Khanh
    Commented May 22, 2023 at 12:50
  • $\begingroup$ I don't know your one-way model. Can you give more details, perhaps in your question? $\endgroup$
    – Ute
    Commented May 22, 2023 at 17:10
  • $\begingroup$ It's just the Center mean + Random error (CE model) for multiple treatments: $Y = \mu_{i} + \epsilon_{ij}, i=0, 1, ..., k$ (number of treatments) and $j=0, 1,..., n_{i}$ where $n_{i}$ is the number of observations in treatment $i$, $\epsilon_{ij} \sim N(0, \sigma^2)$. Also, the book is available online. $\endgroup$
    – Tran Khanh
    Commented May 23, 2023 at 1:02
  • $\begingroup$ I need a bit more information to map your case of multiple treatment comparison to my answer in a helpful manner: What is the test you are looking for? What is the null hypothesis? (Sorry, I don't have time to find a book online and read it :-)) $\endgroup$
    – Ute
    Commented May 23, 2023 at 11:23
  • $\begingroup$ The $H_{0}$ for the one-way model is all treatment (or population) means are equal, $H_{a}$ at least two means are not equal. And actually in the your example, what confounds me is how to construct hypothesis: There're 100 variables (height, weight, bmi,...) and the two locations (A, B), so I guess it's two-way models. But I'm currently stuck on the Bonferroni comparison for the one-way model. $\endgroup$
    – Tran Khanh
    Commented May 24, 2023 at 5:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.