I understand the intuition behind the MCP but I'm having trouble pinpointing exactly the cause, what is it that should be avoided, or at least accounted for.
In its most blunt definition, I agree that if I take any data and apply a brute force approach to it trying every possible null hypotheses, I'll eventually find one that can be rejected with an arbitrary alfa (e.g., 5%) and declare a discovery.
But in many definitions of MCP I read something like "the more you test the more you are likely to find", and although I agree, I don't necessarily see it as a problem (or at least the root of the problem). For example, if many researchers are analyzing the same phenomenon with the same available data, each testing its own hypothesis, it's more likely that one will reach a discovery (than if it were just one researcher), does that mean that they should be applying some type of correction to their target alfa (e.g., a Bonferroni correction)? I'm assuming the answer is no, but then it doesn't become clear why should a single researcher testing many hypotheses should (again, agreeing that the testing system can be abused and there should be a correction for that).
When does this increased chance to find a discovery (reject a null hypothesis) become a problem? When thinking about the causes there are some factors that come to mind, but I'm not sure which one of them (or others not listed here) is more related to the cause of this problem:
Post hoc analysis: I understand that the hypotheses should be (preferably) formulated a priori, if not, I'm just looking at the data trying to guess which hypothesis I could fit under the desired alfa.
Reusing data: Is the problem gone if I use different data sets for each hypothesis I test? The chance of finding a discovery will still increase the more hypotheses I test (even on different data sets).
Independent researchers: reusing the previous example, is the MCP related to the same research team/effort? Or it applies to multiple independent researchers working on the same problem (or even on the same or similar data)?
Independent hypotheses: related to the previous issue, does the problem arise (or is more strongly manifested) when the hypotheses are independent? (because I'm covering more of the search space) or the main issue is trying similar hypotheses with small variations (e.g., fine-tuning a parameter)?
I could summarized the points above, in my interpretation, as (1) and (2) being forms of reducing the search space (borrowing terminology from optimization theory) where I'm making it easier to find a discovery; and (3) and (4) as using more orthogonal search methods that cover more of this search space every time they are applied (i.e., every time a hypothesis is tested). But these are just some possible causes I could come up with, to help get an answer started, there is much more I am missing I'm sure.
This question is somewhat of a follow up from a previous one that asks why is multiple comparison a problem, raising an issue similar to the distinction between the FWER and the FDR (if I understand the question correctly). In this question I don't regard that as an issue (although I would be more inclined to use FDR), both rates imply that there is a problem when analyzing more than one hypothesis (but I fail to see the distinction from the case when I analyze different unrelated problems, finding a discovery for each one of them with 5% significance, which means that when I've "solved" 100 problems rejecting null hypotheses, 5 of them -expected value- would probably be wrong). The best answer to that question implied that there was not a definite answer to it, and maybe there isn't one for this question either, but it would still be very helpful (to me at least) to elucidate as much as possible where is the cause of the MCP error coming from.
(Another answer to the same question suggested a paper that explains the benefits of the Bayesian multilevel model perspective over the classical perspective. This is another interesting approach worth investigating but the scope of this question is the classical framework.)
There are already several questions about this problem, many worth reading (e.g., 1, 2, 3, 4) which address (from different perspectives) the issues raised above, but I still feel a more unified answer (if that is even possible) is lacking, hence this question, which I hope doesn't decrease the (already problematic) SNR.