I have recently been introduced to to Hommel Hochberg corrections. I am trying to find a simple explanation about what this actually is/does, but am having no luck. Can anyone please give a brief and simple description about Hommel Hochberg corrections?

  • 2
    $\begingroup$ Where have you been introduced to the Hommel Hochberg corrections, if you don't mind my asking? I have never seen a paper put together by the two of them. They each have their own methods along with some work done with others (i.e. Benjiamini-Hochberg) but I have not seen them together. Maybe you meant them separately? $\endgroup$ – Cristian Dima Sep 30 '13 at 12:10
  • $\begingroup$ Thanks for your response. My supervisor asked me to use them for a study, in the following context...Hommel-Hochberg corrections were applied to lower α levels for repeated measures. Perhaps they meant separately, but she's only discussed them as one! $\endgroup$ – Bruce Rawlings Sep 30 '13 at 13:58

I still don't get what your supervisor meant by Hommel-Hochberg seeing that I can't find any such collaboration but I guess it's no harm in putting some useful information out there regarding multiple test procedures.

Introduction. Bonferroni correction

First off, if you know nothing about multiple test procedures you should start by reading about the Bonferroni correction. It's super easy to understand and will give you a good starting base. All that Bonferroni does is to adjust the $\alpha$ value of interest by dividing it by $n$ (total number of alternative hypotheses). So you will end up rejecting any $H_i$ having

$$ p_i < \frac{\alpha}{n} $$

This will keep the family wise error rate below $\alpha$. To give you a sense of how this works imagine you got 20 false alternative hypotheses and you're testing at a significance level $\alpha = 0.05$. Under these conditions, the probability of wrongly rejecting at least one null hypothesis (type I error) is given by

$$ P(\text{type I}) = 1 - P(\text{No type I}) = 1 - (1-0.05)^{20} = 1 - 0.36 = 0.64 $$

So even though you have 20 false alternatives there's a 64% chance you will favor one of them over the null. Using the Bonferroni correction, however, reduces this to

$$ P = 1 - \left(1-\frac{0.05}{20}\right)^{20} = 1 - 0.95 = 0.05 $$

Anyways, this is quite a long piece on Bonferroni when the question is not even about it. It should help you understand however the purpose of the next generation of multiple testing methods which use a step-up procedure. The problem with Bonferroni is that it becomes quite rigid when there's a large number of hypotheses tested and it assigns the same $\omega = \alpha/n$ value to every hypothesis. Step-up procedures work better than Bonferroni because they rank each hypothesis according to its p-value and then assign it a different $\omega$.


Hochberg (1988) presents one step-up procedure. There are others, some even more recent, that you could also look into such as Holm-Bonferoni or Benjamini-Hochberg (1995). The original Hochberg, however, the one you're interested in works like this:

  1. Order the p-values $P(1), P(2), ..., P(n)$ and their associated hypotheses $H(1), ..., H(n)$
  2. Reject all hypotheses $H(k)$ having $P(k) \le \frac{\alpha}{n+1-k}$ where $k = 1, ..., n$

As you can see, unlike the Bonferroni correction, Hochberg's step-up method compares each p-value with a different number. The smaller p-values get compared to lower numbers and the higher p-values get compared to higher numbers. This is the "correction" you are looking for.

Note that the Holm method I linked above is also referenced in Hochberg's paper so you might want to check that one out as well - they're very similar. Holm's btw, it's actually a step-down procedure. You can figure out the difference on your own I'm sure. Another quite important paper about both Hochberg and (next up) Hommel reference is Simes (1986). You should really check this one out as well to better understand the two methods.


Hommel's method is more powerful than Hochberg but is kind'of more difficult to compute and wrap your head around. The shortest and easiest explanation I could find was in Multiple Hypothesis Testing (1995) (great multiple test procedures review btw) and it goes like this:

Let $j$ be the largest integer for which $$p_{n - j + k} > \frac{k\alpha}{j}$$ for all $k = 1, ..., j$.

If no such $j$ exists, reject all hypotheses; otherwise, reject all $H_i$ with $p_i \le \frac{\alpha}{j}$. Both $j$ and $i$, btw, go from $1$ to $n$.

The original paper, which you should really look into for a deeper understanding is Hommel (1988). Note that there are various assumptions each of these methods make, various differences between them, and different capabilities for each method. You should really study the papers to gain a deeper understanding of the subject.


Newer methods you might look into are White (2000) (uses a bootstrap method and as opposed to "correcting" alpha it offers a new way of calculating the p-value) and for an expanded version of White's, Wolf and Romano (2003). These are slightly different methods so they may not be relevant to you but they are quite powerful for testing multiple models against the same data (null hypothesis).

Sorry if some of my text was a bit off-topic. I got into this subject recently and I kind'of like writing about it. Hope this is helpful. Let me know if you actually find a Hommel-Hochberg method as I have not been able to.

| cite | improve this answer | |
  • 2
    $\begingroup$ Nice answer (+1). One detail: Perhaps you were trying to draw a connection to the Benjamini Hochberg procedure and its assumptions, but the section on Bonferroni correction is implicitly assuming independent tests, which is unnecessary and, in a sense, misleading. I would argue that showing the general case is actually more enlightening simply because it readily accords with common-sense notions and also shows, in a certain sense, why you need stronger assumptions to get a procedure with strictly better performance. $\endgroup$ – cardinal Sep 30 '13 at 19:23
  • $\begingroup$ I am taking the liberty of correcting your Hommel procedure "otherwise, reject all $H_{i}$ with $p_{i}\le \frac{k\alpha}{j}$" should be "otherwise, reject all $H_{i}$ with $p_{i}\le \frac{\alpha}{j}$" per Hommel, (1988, p.384, third-to last sentence of the Closed Test Procedures Section), and also per Shaffer (1995, p.571, last sentence of Hommel's Test Procedure). $\endgroup$ – Alexis Sep 5 '14 at 1:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.