If I run one experiment, and obtain a p-value of $p=.03$ and I have a priori decided that my $\alpha=.05$. I have a significant a result, I conclude that my data reject $H0$.
If I run two identical experiments with the same study $\alpha$, both with a p-value of $p=.03$. Now as I have done two different tests, I correct for multiple comparisons (e.g. Bonferroni), and the $p$ the values become non-significant, as my alpha is divided by the number of tests done.
Imagine the same outcome but for hundred experiments in the same study, testing the same hypothesis.
How can it be that gathering evidence that converges in rejecting the null, makes it less significant and therefore prevents from rejecting it?
I understand that common sense (and for example a Bayesian approach), should find more evidence in rejecting the null. But, I'm more interested in knowing, if my reasoning is flawed (e.g. multiple comparisons should not be applied in this case, but then I'm interested in why) or this is a known flaw, and we just live with it, and the decision is taken based on common sense and not on a strictly mathematical proof.