combined p values from different hypothesis

Question

I am currently working on a project in which I'm simultaneously identifying and quantifying protein in k samples. Experimentally, for each of the k samples, I have n identified proteins, with a p value that gives me the reliability of that identification, and the quantity of each protein in each sample. I don't have much control on how this p value is computed.

Other than that, samples are independent, treatment and analysis is the same.

For each of the n proteins I compute a combined p value from the individual p values for each protein, using either Fisher's or Stouffer's, to assess the significance of identifying each protein in the k samples, and I get a p value for the identification of each protein, call it p1.

Then, I want to know the average quantity of each protein in the 4 samples, and I'm testing to check where H0: measured_average = goal_average, and I get a p value for this for each protein, call it p2.

I've been reading a lot of research papers both on statistics and on the experiments I'm performing, and to the best of my knowledge and readings, combining p values, and even combining combined p values, can only be performed if the statistic tests for each individual p value are the same, which I guess it is not the case.

My question – can I combine p1 and p2? Is there anyway to have an aggregated p value for this situation? Or I just leave it at that?

Answer 1

The view that all the tests have to be the same, although widely believed, is not true. I have provided a detailed account in my answer Test for significant excess of significant p-values across multiple comparisons

To summarise from that

The null hypothesis H0 is well defined, that all pi have a uniform distribution on the unit interval. There are two classes of alternative hypothesis

HA: all pi have the same (unknown) non--uniform, non--increasing density,
HB: at least one pi has an (unknown) non--uniform, non--increasing density.

If all the tests being combined come from what are basically replicates then HA is appropriate whereas if they are of different kinds of test or different conditions then HB is appropriate. Note that Birnbaum specifically considers the possibility that the tests being combined may be very different for instance some tests of means, some of variances, and so on.