Applying multiple test correction for stock prices

Question

I want to work out if there are any associations between the stock prices within two indexes, S&P 500 and FTSE 100. I plan to perform a simple regression for each pair of stocks. I've read that I need to adjust for multiple testing. If I use Bonferroni, I have to divide the significant p-value by $100 \times 505 = 50500$, and I end up in the place that no stocks are associated. Is the multiple testing adjustment applied properly here? It seems strange that there are no associations.

Answer 1

I don't think testing (multiple or simple) is appropriate here at all. Financial theory alone tells us that the associations are real, due to concurrent common underlying factors that drive stock prices. Most associations will be positive, but some will be negative.

Instead of testing, which suggests that you want to identify which associations are non-zero (when all are non-zero in reality), I would suggest a descriptive analysis using the betas. Since all data are returns, you do not have to worry about scaling issues that would make the betas not comparable. Do some graphical analyses, perhaps some clustering analyses, and come up with some interesting patterns. You could look at various thresholds on the betas (eg, betas>1 or .5), but don't bother using any significance criterion to decide the thresholds. In short, come up with some kind of interesting research hypotheses. The zero/nonzero hypotheses are simply not viable here.

You never mentioned sample size. Given the extreme positive associations between stock prices, I would guess your sample size must be pretty small, otherwise you would find significant differences even when using the extreme .05/50500 threshold.

On the other hand, if your tests were all for some form of market inefficiency; eg, do some past returns of some stocks predict future returns of others?, then the extreme Bonferroni threshold .05/50500 is perfectly valid and appropriate. There are slight improvements that can be had by incorporating dependence structures and distributional characteristics (particularly important given the heavy tailed nature of the return distributions), but as a good starting point, Bonferroni is great when testing a large number of hypothesis where there is strong theory that all nulls are true. A great analogy is ESP (extra-sensory perception) experiments: There is strong theory to suggest that ESP really does not exist at all, so if you decided to test 50500 individual hypotheses, where rejection of any one of them would prove existence of ESP, then you should apply a Bonferroni correction.

Like the case of ESP, there is strong theoretical basis to believe that there are no associations when testing individual hypotheses of market inefficiency. (Or, to be more technically correct, that the true effect is within $\epsilon$ of 0, where $\epsilon$ is minutely small).