What are Hommel Hochberg corrections?

Question

I have recently been introduced to to Hommel Hochberg corrections. I am trying to find a simple explanation about what this actually is/does, but am having no luck. Can anyone please give a brief and simple description about Hommel Hochberg corrections?

Answer 1

I still don't get what your supervisor meant by Hommel-Hochberg seeing that I can't find any such collaboration but I guess it's no harm in putting some useful information out there regarding multiple test procedures.

Introduction. Bonferroni correction

First off, if you know nothing about multiple test procedures you should start by reading about the Bonferroni correction. It's super easy to understand and will give you a good starting base. All that Bonferroni does is to adjust the $\alpha$ value of interest by dividing it by $n$ (total number of alternative hypotheses). So you will end up rejecting any $H_i$ having

$$ p_i < \frac{\alpha}{n} $$

This will keep the family wise error rate below $\alpha$. To give you a sense of how this works imagine you got 20 false alternative hypotheses and you're testing at a significance level $\alpha = 0.05$. Under these conditions, the probability of wrongly rejecting at least one null hypothesis (type I error) is given by

$$ P(\text{type I}) = 1 - P(\text{No type I}) = 1 - (1-0.05)^{20} = 1 - 0.36 = 0.64 $$

So even though you have 20 false alternatives there's a 64% chance you will favor one of them over the null. Using the Bonferroni correction, however, reduces this to

$$ P = 1 - \left(1-\frac{0.05}{20}\right)^{20} = 1 - 0.95 = 0.05 $$

Anyways, this is quite a long piece on Bonferroni when the question is not even about it. It should help you understand however the purpose of the next generation of multiple testing methods which use a step-up procedure. The problem with Bonferroni is that it becomes quite rigid when there's a large number of hypotheses tested and it assigns the same $\omega = \alpha/n$ value to every hypothesis. Step-up procedures work better than Bonferroni because they rank each hypothesis according to its p-value and then assign it a different $\omega$.

Hochberg

Hochberg (1988) presents one step-up procedure. There are others, some even more recent, that you could also look into such as Holm-Bonferoni or Benjamini-Hochberg (1995). The original Hochberg, however, the one you're interested in works like this:

1. Order the p-values $P(1), P(2), ..., P(n)$ and their associated hypotheses $H(1), ..., H(n)$
2. Reject all hypotheses $H(k)$ having $P(k) \le \frac{\alpha}{n+1-k}$ where $k = 1, ..., n$

As you can see, unlike the Bonferroni correction, Hochberg's step-up method compares each p-value with a different number. The smaller p-values get compared to lower numbers and the higher p-values get compared to higher numbers. This is the "correction" you are looking for.

Note that the Holm method I linked above is also referenced in Hochberg's paper so you might want to check that one out as well - they're very similar. Holm's btw, it's actually a step-down procedure. You can figure out the difference on your own I'm sure. Another quite important paper about both Hochberg and (next up) Hommel reference is Simes (1986). You should really check this one out as well to better understand the two methods.

Hommel

Hommel's method is more powerful than Hochberg but is kind'of more difficult to compute and wrap your head around. The shortest and easiest explanation I could find was in Multiple Hypothesis Testing (1995) (great multiple test procedures review btw) and it goes like this:

Let $j$ be the largest integer for which $$p_{n - j + k} > \frac{k\alpha}{j}$$ for all $k = 1, ..., j$.

If no such $j$ exists, reject all hypotheses; otherwise, reject all $H_i$ with $p_i \le \frac{\alpha}{j}$. Both $j$ and $i$, btw, go from $1$ to $n$.

The original paper, which you should really look into for a deeper understanding is Hommel (1988). Note that there are various assumptions each of these methods make, various differences between them, and different capabilities for each method. You should really study the papers to gain a deeper understanding of the subject.

Extras

Newer methods you might look into are White (2000) (uses a bootstrap method and as opposed to "correcting" alpha it offers a new way of calculating the p-value) and for an expanded version of White's, Wolf and Romano (2003). These are slightly different methods so they may not be relevant to you but they are quite powerful for testing multiple models against the same data (null hypothesis).

Sorry if some of my text was a bit off-topic. I got into this subject recently and I kind'of like writing about it. Hope this is helpful. Let me know if you actually find a Hommel-Hochberg method as I have not been able to.