How can we infer from the combined p-value of Fisher's method whether the null hypotheses are all true?

Question

Using the R package oolong (oolong), I used Fisher's method to combine the p-values of three binomial tests. I have difficulty understanding the combined p-value. I understood it to mean that the combined p-value tells me whether all null hypotheses were true. What exactly needs to be assessed for this? Reading this Understanding Fisher's combined test, I have two guesses:

1. If the combined p-value is below the significance level chosen for the individual binomial tests, the null hypotheses for at least one test can be rejected.
2. Since Fisher's method is based on the chi-square distribution, the critical value must be calculated depending on the degrees of freedom, as in the chi-square test. If the combined p-value is above the critical value, the null hypotheses for each test can be rejected.

I know that this is a very basic question, but unfortunately I don't know much about statistics and would be really grateful if someone could help me. Any links to tutorials or papers would also be greatly appreciated.

Answer 1

There are several methods for combining p-values and all work (approximately) only in the circumstance where more than one dataset are available to interrogate a singular null hypothesis. (As @Sextus Empiricus said.) The combined p-value gives you an (approximate) index of how strongly the data in combination argue against that single null hypothesis as a significance test. You can also use the relationship between the combined p-value and a predetermined threshold to 'decide' whether to reject the null hypothesis if you prefer the hypothesis test approach.

If you were to 'combine' p-values from tests of differing null hypotheses there certainly would be reason to wonder at what exactly is being tested.