# Multiple hypothesis tests before and after data collection

My professor asked the following problem to drill in the ideas of significance levels and $$p$$-values during lectures and it's really doing my head in. Suppose I want to test a particular hypothesis $$H_1$$. Is there a difference between the two hypothesis testing scenarios?

1. I pose 99 additional hypotheses $$\{H\}_{2\leq i\leq 100}$$, collect data, fail to reject null hypothesis on $$\{H\}_{1\leq i\leq 99}$$  at a 0.05 significance level, then get a $$p$$-value of 0.01 for $$H_{100}$$.

2. I collect data, fail to reject null hypothesis on $$H_1$$ at a 0.05 significance level, pose a new hypothesis $$G$$ calculated on the collected data, immediately obtain a $$p$$-value of 0.01.

What can I claim about $$H_{100}$$ and $$G$$ at a 0.05 significance level or a 0.001 significance level? I'm comfortable with my understanding of when to reject for a single test, but when it comes to combining multiple hypothesis tests together I lose track of what we can and cannot do. Could someone please provide some insight into this? Will be eternally grateful!

• It depends strongly on whether you want to evaluate the evidence concerning H1 with a significance test or whether you want to decide whether to reject H1. See this for a little more essential background: stats.stackexchange.com/questions/16218/… – Michael Lew Nov 7 '20 at 20:27
• If you are going to perform a Neyman-Pearsonian hypothesis test then you must set the critical value (i.e. the "significance level") in advance of performing the test. You really need read a little more about hypothesis tests and significance tests before attempting to sort out anything as complicated as your question. – Michael Lew Nov 8 '20 at 4:03